-- Logs begin at Tue 2022-10-11 06:03:10 UTC, end at Tue 2022-10-11 07:28:55 UTC. -- Oct 11 06:28:09.410063 dv-cinder-854954-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 11 06:28:09 2022] *** Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: nodename: dv-cinder-854954-3-iscsi Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: machine: x86_64 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: clock source: unix Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: pcre jit disabled Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: detected number of CPU cores: 4 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: current working directory: / Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: detected binary path: /usr/bin/uwsgi-core Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: your processes number limit is 47680 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: your memory page size is 4096 bytes Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: detected max file descriptor number: 2048 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: lock engine: pthread robust mutexes Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: thunder lock: enabled Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 11 06:28:09.430099 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 11 06:28:09.462901 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: Python main interpreter initialized at 0x564e2f6689c0 Oct 11 06:28:09.462901 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: python threads support enabled Oct 11 06:28:09.462901 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: your server socket listen backlog is limited to 100 connections Oct 11 06:28:09.462901 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: your mercy for graceful operations on workers is 90 seconds Oct 11 06:28:09.463248 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: mapped 403077 bytes (393 KB) for 2 cores Oct 11 06:28:09.463374 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: *** Operational MODE: preforking *** Oct 11 06:28:09.463649 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: *** uWSGI is running in multiple interpreter mode *** Oct 11 06:28:09.463649 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: spawned uWSGI master process (pid: 81471) Oct 11 06:28:09.463849 dv-cinder-854954-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 11 06:28:09.464851 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 1 (pid: 81484, cores: 1) Oct 11 06:28:09.465171 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 2 (pid: 81485, cores: 1) Oct 11 06:28:09.466407 dv-cinder-854954-3-iscsi devstack@keystone.service[81471]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 11 06:28:13.747855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 06:28:13.747855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: from cryptography.utils import int_from_bytes Oct 11 06:28:13.748591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 06:28:13.748591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: from cryptography.utils import int_from_bytes Oct 11 06:28:13.749324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 06:28:13.749324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: from cryptography.utils import int_from_bytes Oct 11 06:28:13.749771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 06:28:13.749771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: from cryptography.utils import int_from_bytes Oct 11 06:28:13.798402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 11 06:28:13.798588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 11 06:28:13.798780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] command line args: [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 11 06:28:13.799042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 11 06:28:13.799215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ================================================================================ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 11 06:28:13.799480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] admin_token = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.799677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 11 06:28:13.799771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config_dir = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.799902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 11 06:28:13.799988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.800096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] command line args: [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 11 06:28:13.800179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] config_source = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.800285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] debug = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.800368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 11 06:28:13.800480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.800735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ================================================================================ {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 11 06:28:13.800799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] admin_token = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.800866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.800994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] config_dir = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] config_source = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] debug = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.801904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_config_append = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_dir = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.802901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_config_append = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_dir = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_file = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.803935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.804930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.805830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] publish_errors = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.806879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] publish_errors = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] standard_threads = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.807949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_journal = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_json = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_stderr = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] use_syslog = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] standard_threads = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.809438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.809614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] use_journal = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.809816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] use_json = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.809949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] use_stderr = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.810025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.810124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] use_syslog = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.810209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.810354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 06:28:13.810435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.810608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.810699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.810845 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.810935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.811921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.812883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.813983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.814968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.815053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.815196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.815792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.815961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.816983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.817916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.818996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.819933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.820920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.821994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.822914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.823912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.824981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.825925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.826940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.827888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828590 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.828981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.829925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.830995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.831689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.831786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.831927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.832912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.833983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.834995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.external = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.835982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.password = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] auth.token = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.836901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.external = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.837848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.password = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] auth.token = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.838931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.839974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.840938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.841894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.842997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.843954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.844984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.845920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.846875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847357 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.847982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.848938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.849955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.850877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.851978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852918 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.852996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.853907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.854988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.855960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.856855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.857946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.858965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.859916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.860859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861019 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.861953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.862986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.863950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.864879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.865968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.866990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.867969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.868886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] role.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.869994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] role.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.870931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.871923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.872900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.873990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.874921 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.875986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876367 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.876902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.877055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.877132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.877298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.877379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.877530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.877608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.878987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879481 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.879957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880909 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.880995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.881948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.882887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.connection = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.883988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.884924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.885936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.886010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81485) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 11 06:28:13.886081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.886314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.886557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.886787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.887027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.887261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 06:28:13.887460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 11 06:28:13.915527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.915753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.915976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.916497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.916731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.916957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.924262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.925836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.952397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.952622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.953459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:13.953693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.005699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.006321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.054064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.054291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.055246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.055504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.209588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.209805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.210855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.211073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.211289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.233417 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.233642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.233861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.233943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.234964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.235178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.246293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.246712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.258878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259590 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.259923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.260018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.260131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81485) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.260213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.260345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.260558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 06:28:14.277045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.277247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 06:28:14.277585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.277667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.277784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 06:28:14.277867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 06:28:14.278059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.278142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.278320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 06:28:14.278401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.278575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.278665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.278832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.278914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.279100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.279362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.288863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.289145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.289415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.289526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.289697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.289808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.290085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.290368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.296651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.297585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.299553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.300354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.300636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.300920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.301203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.301995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.302990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.303260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.303558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.303833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.325219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.326398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.326665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.326949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.327892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.328186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.332304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.332771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.333240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.333357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.333648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.333934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.340292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.340587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.341286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.341573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.346223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.346541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.347351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.347664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.347746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.347942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.348843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.349074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.349341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.349609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.364238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.364526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.365164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.365442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.368404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.369295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.371664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.371966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.372542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.372837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.376197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.377024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.379332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.380327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.382728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.383033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.383321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.383563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.383646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.383863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.383946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.384148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.384232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.384427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.384703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.384980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.400751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.401051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.401198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.401322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.401485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.401593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.401752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.402013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.408723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.409158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.410347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.410698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.411898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.412180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.412296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.412515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.414844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.415133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.415218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.415434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.415518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.415719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.415803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.415997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.416109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.416390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.427920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.428016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.428211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.428294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.428496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.428592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.428784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.428867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.429159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.429241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.429438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.429522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.429717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.429799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.445060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.445164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.447974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.448064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.450828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.450939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.452356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.452483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.453811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.454062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.454145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.454351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.454491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.454758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.460836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.461172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.462277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.462635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.465176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.465545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.465636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.465836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.465943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.466114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.466234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.466517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.474397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.474683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.474826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.474958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.475130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.475434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.481815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81485) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.482098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.482302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 06:28:14.482385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.482593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.482675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.482879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.482970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.483995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.484954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.485106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.485186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.485374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81485) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.485455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.485640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.485942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 06:28:14.519781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564e2f6689c0 pid: 81485 (default app) Oct 11 06:28:14.520643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564e2f6689c0 pid: 81484 (default app) Oct 11 06:28:14.522975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683433b-31a9-45ed-bbbb-bbe769115a2a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:14.523259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683433b-31a9-45ed-bbbb-bbe769115a2a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:14.523559 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683433b-31a9-45ed-bbbb-bbe769115a2a None None] PATH_INFO: `/v3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:14.527691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Tue Oct 11 06:28:14 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:21.270811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:21.271321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:21.271443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:21.275658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:28:21.281275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:21.281892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:21.281892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:21.352082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: INFO dbcounter [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] Registered counter for database keystone Oct 11 06:28:21.370626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG oslo_db.sqlalchemy.engines [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81485) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 11 06:28:21.376949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writer thread running {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 11 06:28:21.556017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG passlib.handlers.bcrypt [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81485) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 11 06:28:21.556322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG passlib.handlers.bcrypt [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81485) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 11 06:28:21.614262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:21.615919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:21.744750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 06:28:22.174942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:22.245015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: INFO dbcounter [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] Registered counter for database keystone Oct 11 06:28:22.266023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG oslo_db.sqlalchemy.engines [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81484) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 11 06:28:22.272300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writer thread running {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 11 06:28:22.391704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e43ba-7e48-4b46-a757-256956037d1f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:22.396509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:22.397132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:22.397415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:22.397690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:22.406324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:22.458489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING keystone.common.rbac_enforcer.enforcer [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 11 06:28:22.458770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e4e29-f05e-40f0-b9d4-95b73600702a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:22.468314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Oct 11 06:28:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:28:22.474134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:22.491641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eec599-7e05-414c-aafd-dd9b869fbcf3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:22.496080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:22.497180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:22.497465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:22.497737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] PATH_INFO: `/v3/projects/4b69df7fd16f47b18a624f52e5487b78` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:22.499340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] RBAC: Authorizing `identity:get_project(project_id=4b69df7fd16f47b18a624f52e5487b78)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:22.552556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 11 06:28:22.552905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f452630-4ecc-4f7b-b523-44e8bedfaa16 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:22.557351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Oct 11 06:28:22 2022] GET /identity/v3/projects/4b69df7fd16f47b18a624f52e5487b78 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:24.680596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153bef-7483-45d7-97d7-b2e171539a4f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:24.681093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153bef-7483-45d7-97d7-b2e171539a4f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:24.681093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153bef-7483-45d7-97d7-b2e171539a4f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:24.685133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:24.691152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:24.691541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:24.691745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:24.711792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:24.713309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:24.721083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 06:28:24.987501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153bef-7483-45d7-97d7-b2e171539a4f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:24.999232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153bef-7483-45d7-97d7-b2e171539a4f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:25.003876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47864870-01c2-489a-96d0-2a0f743c974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:25.004465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47864870-01c2-489a-96d0-2a0f743c974c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:25.004740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47864870-01c2-489a-96d0-2a0f743c974c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:25.005009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47864870-01c2-489a-96d0-2a0f743c974c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:25.012389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47864870-01c2-489a-96d0-2a0f743c974c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:25.014143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47864870-01c2-489a-96d0-2a0f743c974c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:25.023137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Oct 11 06:28:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:28:25.028258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:25.038052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea3967-6412-4e78-a97e-bec079f24278 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:25.042412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9164d-bede-4dc4-aa48-17333198f6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:25.043030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9164d-bede-4dc4-aa48-17333198f6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:25.043312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9164d-bede-4dc4-aa48-17333198f6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:25.043626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9164d-bede-4dc4-aa48-17333198f6e9 admin admin] PATH_INFO: `/v3/users/cdd55bd4e43346b29310a317dad7a19b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:25.044938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9164d-bede-4dc4-aa48-17333198f6e9 admin admin] RBAC: Authorizing `identity:get_user(user_id=cdd55bd4e43346b29310a317dad7a19b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:25.046765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9164d-bede-4dc4-aa48-17333198f6e9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:25.048311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Tue Oct 11 06:28:25 2022] GET /identity/v3/users/cdd55bd4e43346b29310a317dad7a19b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:27.213312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860eb0b-fc05-4602-8fe8-382b1d18f6fd None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.213830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860eb0b-fc05-4602-8fe8-382b1d18f6fd None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.213830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860eb0b-fc05-4602-8fe8-382b1d18f6fd None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.215185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:27.220266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.220654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.220942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.240353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:27.241532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:27.254338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d88de-d721-4ee8-bdd8-0253084323b5 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.254702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d88de-d721-4ee8-bdd8-0253084323b5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.254987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d88de-d721-4ee8-bdd8-0253084323b5 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.256340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:28:27.261382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.261666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.261935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.304162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG passlib.handlers.bcrypt [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81484) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 11 06:28:27.304544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG passlib.handlers.bcrypt [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81484) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 11 06:28:27.322426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:27.353175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:27.354741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:27.452862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:27.586214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:27.610023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:27.689414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86374-0426-436f-8b18-ba5c49e99059 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:27.719432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:27.722650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:27.723288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.723605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.723880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.736232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:27.745122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:27.746071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:27.746889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:27.751895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:27.821982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249d06b-ef26-44a7-9b42-65d6bddb352e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:27.824235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:27.832371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:27.851362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68fdd3-ee1a-4b67-9518-5e7049392031 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:27.859034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:27.859925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.860209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.860483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.861453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:27.862921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:27.874466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.sql.core [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 06:28:27.874466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 06:28:27.874466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [parameters: {'id': '63f45eda6b3e48a5935e9fd66958a11f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 06:28:27.874466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81484) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 06:28:27.885149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:27.885763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:27.886046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:27.886319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:27.886994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:27.887353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:27.888790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:28:27.892642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result.read() Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 06:28:27.893510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application yield resource Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise self.value Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application session.commit() Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 06:28:27.895116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.session.flush() Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application compat.raise_( Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 06:28:27.896561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 06:28:27.897562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result.read() Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:28:27.898454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [parameters: {'id': '63f45eda6b3e48a5935e9fd66958a11f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 06:28:27.899408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:27.900288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 8/15] 10.222.0.39 () {60 vars in 1137 bytes} [Tue Oct 11 06:28:27 2022] POST /identity/v3/domains => generated 190 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:28:27.902184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:27.903970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 8/16] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:27.910140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:28.000573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7666f-4e72-4015-8f7b-95ac344aeba0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:28.006708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33542-39fe-47c5-ad42-dd7c788e9d65 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:28.033669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:28.035038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:28.035328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:28.035637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:28.039901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:28.040528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:28.040810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:28.041078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] PATH_INFO: `/v3/users/cdd55bd4e43346b29310a317dad7a19b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:28.042200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd55bd4e43346b29310a317dad7a19b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:28.043682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:28.045243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:27 2022] GET /identity/v3/users/cdd55bd4e43346b29310a317dad7a19b => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:28.051225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:28.054164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:28.055648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:28.064881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:28:28.071685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:28.072712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:28:28.073686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:28:28.073686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:28:28.073686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:28.073686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:28.078901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:28.102113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:28.153966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c65e1-39be-4ce6-86d6-1c98facd7396 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:28.178955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae8172-5c98-43d7-9fff-719c4f372246 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:28.189167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:28.189799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:28.190075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:28.190343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:28.191514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:28.192955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:28.194488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:28.199986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:28.215345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:28.216076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:28.216427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:28.216775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:28.217495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:28.217892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:28.219345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b3f2f-5670-46f0-bdcb-2924bb92a426 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:28.237142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:28.281548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:28.302340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61b31-906b-4087-a1ad-7d3b58be260d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:28.337539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0288e6-a78d-4e64-bf6d-3ad3e59a19b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:28.338212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0288e6-a78d-4e64-bf6d-3ad3e59a19b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:28.338488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0288e6-a78d-4e64-bf6d-3ad3e59a19b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:28.338759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0288e6-a78d-4e64-bf6d-3ad3e59a19b9 None admin] PATH_INFO: `/v3/domains/default/users/cdd55bd4e43346b29310a317dad7a19b/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:28.340737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0288e6-a78d-4e64-bf6d-3ad3e59a19b9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cdd55bd4e43346b29310a317dad7a19b, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:28.342200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0288e6-a78d-4e64-bf6d-3ad3e59a19b9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:28.459861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 11/21] 10.222.0.39 () {58 vars in 1361 bytes} [Tue Oct 11 06:28:28 2022] PUT /identity/v3/domains/default/users/cdd55bd4e43346b29310a317dad7a19b/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:28:30.804439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe7e18-6d8b-426f-8969-f06e6d3a7357 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:30.804917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe7e18-6d8b-426f-8969-f06e6d3a7357 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:30.804917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe7e18-6d8b-426f-8969-f06e6d3a7357 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:30.806688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:28:30.812400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186e87-8f89-4ca7-8a69-305161cc66b9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:30.812791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186e87-8f89-4ca7-8a69-305161cc66b9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:30.812982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186e87-8f89-4ca7-8a69-305161cc66b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:30.840671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0e186e87-8f89-4ca7-8a69-305161cc66b9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:30.842271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0e186e87-8f89-4ca7-8a69-305161cc66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:30.944775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:31.071534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9fc6f-9c6f-4483-8868-f412bd9369e4 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.071816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9fc6f-9c6f-4483-8868-f412bd9369e4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.072046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9fc6f-9c6f-4483-8868-f412bd9369e4 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.073710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 12/24] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:31.082697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.083026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.083298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.086648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6268c8-1f21-4c8d-bde5-8279281b39aa None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.086942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6268c8-1f21-4c8d-bde5-8279281b39aa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.087220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6268c8-1f21-4c8d-bde5-8279281b39aa None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.088883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 13/25] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:31.096510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.096843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.097596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.106726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:31.107975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:31.129064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:31.130297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:31.212577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:31.220979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 13/26] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:31.271942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 14/27] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:31.355212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:31.500188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ee467-7617-46c1-aa28-cfdace0641a1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:31.549520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:31.550203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.550489 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.550762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.561799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:31.563532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:31.576577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:31.600169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:28:31.606011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:31.606845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:28:31.607664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:28:31.607664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:28:31.607664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:31.607664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 14/28] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:31.613943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:31.718190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:31.753005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc015890-c8f9-4936-a5e7-cfe47b822a4f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:31.776136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc800f-3544-473c-8959-50c7ca148f53 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:31.795360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a49c78-2685-4af7-a0ab-0f64b2d8d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:31.796148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a49c78-2685-4af7-a0ab-0f64b2d8d265 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.796436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a49c78-2685-4af7-a0ab-0f64b2d8d265 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.796709 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a49c78-2685-4af7-a0ab-0f64b2d8d265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.809255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a49c78-2685-4af7-a0ab-0f64b2d8d265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:31.816421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-c4a49c78-2685-4af7-a0ab-0f64b2d8d265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:31.816859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:31.817630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:31.818347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 15/29] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:28:31.819441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f99e-fbde-4485-ba17-9d0484fbc732 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.819711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f99e-fbde-4485-ba17-9d0484fbc732 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.820018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f99e-fbde-4485-ba17-9d0484fbc732 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.820665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.821136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.821582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 16/30] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:31.821807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.822342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9f99e-fbde-4485-ba17-9d0484fbc732 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:31.822858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:31.823355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:31.825000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:31.851859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 15/31] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:31.853447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:31.899165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:31.935083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9f99e-fbde-4485-ba17-9d0484fbc732 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:31.990974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:31.991619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:31.991897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:31.992164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:31.992852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:31.995041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.011345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a4332-ea99-4e53-8935-182ce0e523e7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:32.044002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:32.069421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:32.070100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.070382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.070654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.071384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:32.071741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:32.073192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc28dfe-e3b9-4cb5-895a-48ded763a388 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.083629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 16/32] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:32.085045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.085333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.085657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.104155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:32.105472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:32.173519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 17/33] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Oct 11 06:28:31 2022] POST /identity/v3/roles => generated 219 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:32.179932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:32.196335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 17/34] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:32.197583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:32.337624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731e72e-dbbd-4a57-ad3d-f732b047a7c9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:32.340569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:32.371815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ead6d-cdcd-4165-853a-8c8029172dbd None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:32.386277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82ab4b-dea9-418b-aa3e-2e9b0c364f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:32.387278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82ab4b-dea9-418b-aa3e-2e9b0c364f8a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.387786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82ab4b-dea9-418b-aa3e-2e9b0c364f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.388345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82ab4b-dea9-418b-aa3e-2e9b0c364f8a None admin] PATH_INFO: `/v3/users/cdd55bd4e43346b29310a317dad7a19b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.392233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82ab4b-dea9-418b-aa3e-2e9b0c364f8a None admin] RBAC: Authorizing `identity:get_user(user_id=cdd55bd4e43346b29310a317dad7a19b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:32.394266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82ab4b-dea9-418b-aa3e-2e9b0c364f8a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.395999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 18/35] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:32 2022] GET /identity/v3/users/cdd55bd4e43346b29310a317dad7a19b => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:32.397371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbbac3-1a9a-47f6-939e-a91f1f80093a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.397655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbbac3-1a9a-47f6-939e-a91f1f80093a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.397928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbbac3-1a9a-47f6-939e-a91f1f80093a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.399517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 19/36] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:32.409084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.411478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.411478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.423926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:32.432898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:32.434105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.434962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.435603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.436531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:32.438489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.460469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:32.461968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:32.471399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 18/37] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:28:32 2022] POST /identity/v3/projects => generated 301 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:32.472745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:32.655695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220d7ea-9463-42ea-a177-463045b4b3c7 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:32.703525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 20/38] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:32.703525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:32.715784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:32.716432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.716720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.717126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.718406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:32.720413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.722023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 19/39] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:32.728039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:32.799308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:32.823143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abee92-8f66-4385-af5d-3ea7963d23fb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:32.825607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:32.863864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50f14d-b00c-4172-ac0e-c53a8269b4c4 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:32.865578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185948a6-4f04-4e4b-a053-219d51b9bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:32.866902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185948a6-4f04-4e4b-a053-219d51b9bb55 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.866902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185948a6-4f04-4e4b-a053-219d51b9bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.866902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185948a6-4f04-4e4b-a053-219d51b9bb55 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.867457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185948a6-4f04-4e4b-a053-219d51b9bb55 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:32.868862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185948a6-4f04-4e4b-a053-219d51b9bb55 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.886255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 21/40] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Oct 11 06:28:32 2022] POST /identity/v3/roles => generated 225 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:32.887722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8115af-3c75-4ac1-90f0-ead211ced4b5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.888178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8115af-3c75-4ac1-90f0-ead211ced4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.888689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8115af-3c75-4ac1-90f0-ead211ced4b5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.890137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 22/41] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:32.897214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.897529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.897775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.909831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:32.910893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:32.911522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:32.912048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:32.913001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.domain.id=default, user.id=cdd55bd4e43346b29310a317dad7a19b) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:32.913533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:32.915392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:32.917766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:32.919165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:32.927404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 20/42] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Oct 11 06:28:32 2022] GET /identity/v3/role_assignments?user.id=cdd55bd4e43346b29310a317dad7a19b&scope.domain.id=default&role.id=287ae34730df480e9b5436bfab5d48bd => generated 530 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:32.992167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:33.012457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 23/43] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:33.122312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937640d8-e754-44de-bdba-b68d726ec78d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:33.139438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:33.163204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:33.165577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:33.165577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:33.165577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:33.165577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:33.166454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:33.185399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 21/44] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Oct 11 06:28:32 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:28:33.294043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:33.386406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:33.396251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ad5f-ac60-4752-90e6-40b665256609 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:33.432855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43482125-72ce-495f-98b1-b906c0ababbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:33.433491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43482125-72ce-495f-98b1-b906c0ababbf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:33.433772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43482125-72ce-495f-98b1-b906c0ababbf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:33.434047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43482125-72ce-495f-98b1-b906c0ababbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:33.435133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43482125-72ce-495f-98b1-b906c0ababbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:33.436639 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43482125-72ce-495f-98b1-b906c0ababbf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:33.438207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 24/45] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:33.443771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:33.540121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1583c2-f56b-45d4-89aa-4205b57af46a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:33.572110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164afa7-b43f-42d8-bd56-0b56862ae1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:33.572656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164afa7-b43f-42d8-bd56-0b56862ae1d5 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:33.572940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164afa7-b43f-42d8-bd56-0b56862ae1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:33.573217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164afa7-b43f-42d8-bd56-0b56862ae1d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:33.573879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164afa7-b43f-42d8-bd56-0b56862ae1d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:33.575699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164afa7-b43f-42d8-bd56-0b56862ae1d5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:33.601760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 22/46] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:28:33 2022] POST /identity/v3/projects => generated 312 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:28:35.705282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef073d60-d8ea-4128-88d1-986b357c847c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:35.705778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef073d60-d8ea-4128-88d1-986b357c847c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:35.705778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef073d60-d8ea-4128-88d1-986b357c847c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:35.707745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 25/47] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:35.713112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:35.713510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:35.713652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:35.731311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:35.732821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:35.751856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:35.796639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 23/48] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:36.060723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef073d60-d8ea-4128-88d1-986b357c847c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:36.160307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef073d60-d8ea-4128-88d1-986b357c847c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:36.192943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9d9ea-0e22-4db0-82f3-7efafc2dd832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:36.193560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9d9ea-0e22-4db0-82f3-7efafc2dd832 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:36.193840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9d9ea-0e22-4db0-82f3-7efafc2dd832 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:36.194113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9d9ea-0e22-4db0-82f3-7efafc2dd832 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:36.195219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9d9ea-0e22-4db0-82f3-7efafc2dd832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:36.196692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9d9ea-0e22-4db0-82f3-7efafc2dd832 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:36.198248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 26/49] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:36.203777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:36.297030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ea150-0c82-43aa-8487-faa562ee9a5f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:36.328823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a6197-1b2b-42ff-a093-c2f9ff04966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:36.329356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a6197-1b2b-42ff-a093-c2f9ff04966a None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:36.329634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a6197-1b2b-42ff-a093-c2f9ff04966a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:36.329902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a6197-1b2b-42ff-a093-c2f9ff04966a None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:36.330542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a6197-1b2b-42ff-a093-c2f9ff04966a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:36.332011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a6197-1b2b-42ff-a093-c2f9ff04966a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:36.356351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 24/50] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Oct 11 06:28:36 2022] POST /identity/v3/projects => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:38.478168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c395a-7c67-4069-8be5-f07115cb8792 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:38.478649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c395a-7c67-4069-8be5-f07115cb8792 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:38.478758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c395a-7c67-4069-8be5-f07115cb8792 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:38.480156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 27/51] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:38.485369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:38.485743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:38.485919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:38.504050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:38.505218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:38.507072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:38.570268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 25/52] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:38.838736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c395a-7c67-4069-8be5-f07115cb8792 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:38.866363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:38.937346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c395a-7c67-4069-8be5-f07115cb8792 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:38.969651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa83bea-317b-403f-a2d5-000dbc856d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:38.970266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa83bea-317b-403f-a2d5-000dbc856d11 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:38.970543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa83bea-317b-403f-a2d5-000dbc856d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:38.970813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa83bea-317b-403f-a2d5-000dbc856d11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:38.971957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa83bea-317b-403f-a2d5-000dbc856d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:38.973879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa83bea-317b-403f-a2d5-000dbc856d11 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:38.975424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 28/53] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:38.980753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:39.074664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8b0bf-9879-45ee-9cf1-4d78e2992442 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:39.108365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e37b6b-e558-453f-8f61-e3df790e327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:39.109000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e37b6b-e558-453f-8f61-e3df790e327e None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:39.109281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e37b6b-e558-453f-8f61-e3df790e327e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:39.109559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e37b6b-e558-453f-8f61-e3df790e327e None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:39.110217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e37b6b-e558-453f-8f61-e3df790e327e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:39.111689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e37b6b-e558-453f-8f61-e3df790e327e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:39.125999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:39.138545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 26/54] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:28:38 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:28:42.461089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899bdf-8fcb-48b0-8fd9-3dd74eb4438a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.461586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899bdf-8fcb-48b0-8fd9-3dd74eb4438a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.461683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899bdf-8fcb-48b0-8fd9-3dd74eb4438a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.463002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 29/55] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:42.468537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729ebc5-b304-405c-b233-6cedca20cf1a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.469353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729ebc5-b304-405c-b233-6cedca20cf1a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.469817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729ebc5-b304-405c-b233-6cedca20cf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.521195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346e6ac-68ac-4344-bc1c-10cc2a8ae35e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.521834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346e6ac-68ac-4344-bc1c-10cc2a8ae35e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.522308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346e6ac-68ac-4344-bc1c-10cc2a8ae35e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.523970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 30/56] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:42.530020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126dd47-a308-4720-ac45-88440dcd1a3e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.530632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126dd47-a308-4720-ac45-88440dcd1a3e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.531110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126dd47-a308-4720-ac45-88440dcd1a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.561380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d126dd47-a308-4720-ac45-88440dcd1a3e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:42.562940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d126dd47-a308-4720-ac45-88440dcd1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:42.608989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d729ebc5-b304-405c-b233-6cedca20cf1a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:42.610859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d729ebc5-b304-405c-b233-6cedca20cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:42.665264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 31/57] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:42.666475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e268-3884-4818-8022-530aa4fbe814 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.666762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e268-3884-4818-8022-530aa4fbe814 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.667036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e268-3884-4818-8022-530aa4fbe814 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.668571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 32/58] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:42.673832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.674280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.674686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.692880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:42.694255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:42.705729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 27/59] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:42.715858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:42.761078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b200843-10c4-47ec-8708-2d9b36b9bd48 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.761627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b200843-10c4-47ec-8708-2d9b36b9bd48 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.762065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b200843-10c4-47ec-8708-2d9b36b9bd48 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.763570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 28/60] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:28:42.766052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 33/61] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:42.769534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200988c4-10ff-4318-a79a-8003c945765c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:42.769969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200988c4-10ff-4318-a79a-8003c945765c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:42.770398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200988c4-10ff-4318-a79a-8003c945765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:42.812402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-200988c4-10ff-4318-a79a-8003c945765c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:42.813797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-200988c4-10ff-4318-a79a-8003c945765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:42.901371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 29/62] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:43.030809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.090136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200988c4-10ff-4318-a79a-8003c945765c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.154100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd3121-894c-45dc-bd75-29ed85855e32 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.165302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.194075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.194854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.195504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.195879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.210690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:43.216684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200988c4-10ff-4318-a79a-8003c945765c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:43.217197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:43.218033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:43.218781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 34/63] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:43.219370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.240872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.262890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.263625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.263906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.264178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.275657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:43.280456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:43.281267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:43.282360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 30/64] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:43.283181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.326882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3373c9-eaa3-4999-80cc-479835354c01 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.360036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb8067-e461-4f58-bc37-e3c78f3f8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.360658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb8067-e461-4f58-bc37-e3c78f3f8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.360934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb8067-e461-4f58-bc37-e3c78f3f8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.361208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb8067-e461-4f58-bc37-e3c78f3f8cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.371593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb8067-e461-4f58-bc37-e3c78f3f8cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-87bb8067-e461-4f58-bc37-e3c78f3f8cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:43.376542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:43.377361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:43.378066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 35/65] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:43.379391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb5144-2ae3-4c74-a8ba-0edef9b8438c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.379980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb5144-2ae3-4c74-a8ba-0edef9b8438c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.380046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce57713-6af7-4423-8e7b-a5288ea7a545 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.380112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb5144-2ae3-4c74-a8ba-0edef9b8438c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.381734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 36/66] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:43.384175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb5144-2ae3-4c74-a8ba-0edef9b8438c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.396674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.419031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.419672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.419959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.420233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.420906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:43.421209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:43.422594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:43.434690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 31/67] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:43.435508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.460795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.484379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb5144-2ae3-4c74-a8ba-0edef9b8438c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.517053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.517674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.517951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.518219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.528528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:43.533629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:43.535006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:43.535791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 37/68] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:43.536409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ff9fa-6b5c-40c6-b203-8ac4c87b904d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.536788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.569639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.570240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.570513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.570781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.571462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] RBAC: Adding query filter params (name=member) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:43.571765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:43.573126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb433b-bd05-4eeb-93b6-214d6188c647 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:43.580923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 32/69] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:43.582573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.582910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.583180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.600497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:43.601671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:43.630181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.635621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbadc-ece4-4e5f-aae8-c8a45c9781ab None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.668155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 33/70] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:43.668962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.669088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577062-46c2-4011-a285-431846429f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.669606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577062-46c2-4011-a285-431846429f10 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.669888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577062-46c2-4011-a285-431846429f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.670162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577062-46c2-4011-a285-431846429f10 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.670827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577062-46c2-4011-a285-431846429f10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:43.671131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577062-46c2-4011-a285-431846429f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:43.672547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577062-46c2-4011-a285-431846429f10 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:43.680125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.687186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 38/71] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:43.689137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577062-46c2-4011-a285-431846429f10 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.777072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b34cc-71f6-41d9-9895-c3765f406477 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.806099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577062-46c2-4011-a285-431846429f10 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.811314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.811969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.812251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.812522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] PATH_INFO: `/v3/users/291c9818263642f080c993ec53610ebc` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.828394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] RBAC: Authorizing `identity:get_user(user_id=291c9818263642f080c993ec53610ebc)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:43.829847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:43.835849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 34/72] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/users/291c9818263642f080c993ec53610ebc => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:43.835849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.847830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.848615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.849262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.849544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.850213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:43.850517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:43.851907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:43.860376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 39/73] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:43.860986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:43.865897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.923349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:43.941834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e49d4-a1c5-443b-a65a-6c7d0275e644 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.967142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006bbcb-efdd-4f79-aee9-c526002a1693 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:43.974580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:43.975210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:43.975518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:43.975793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] PATH_INFO: `/v3/users/291c9818263642f080c993ec53610ebc` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:43.976926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] RBAC: Authorizing `identity:get_user(user_id=291c9818263642f080c993ec53610ebc)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:43.978376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:43.979975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 35/74] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/users/291c9818263642f080c993ec53610ebc => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:43.980738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.001756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.002408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.002691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.002968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] PATH_INFO: `/v3/users/cdd55bd4e43346b29310a317dad7a19b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.004218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd55bd4e43346b29310a317dad7a19b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.005705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.007297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 40/75] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/users/cdd55bd4e43346b29310a317dad7a19b => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.007297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.075760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.084131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac7b15-9bfa-416c-b0f8-02634412a8b2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.104555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e05fd8-fd6d-422f-9f3b-9a0456a26025 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.116750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.117380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.117667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.117943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.119080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.120609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.122248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 36/76] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:43 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.123013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.133576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.141769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.142400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.142682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.142953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] PATH_INFO: `/v3/users/291c9818263642f080c993ec53610ebc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.144621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] RBAC: Authorizing `identity:get_user(user_id=291c9818263642f080c993ec53610ebc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.146075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.152188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 41/77] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:44 2022] GET /identity/v3/users/291c9818263642f080c993ec53610ebc => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.152188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.221049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44707226-9c35-4ee5-bce6-7f8fae2c314f None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.247574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4089d0-b107-4de0-816f-e3dda2767c18 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.254664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.255294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.255610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.255893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.257021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.258465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.260056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 37/78] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.260823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.280685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.281316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.281605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.282131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.283263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.285349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.286980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 42/79] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:44 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.287736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.296214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.347736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.364305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b654785-ea26-4d7b-a302-f1dafbb0bc6b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.385715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1d2b3-f35e-4e91-8c38-35872e3e6276 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.397150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.397783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.398065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.398369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.399526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.400977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.402808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 38/80] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:44 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.403432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.421401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.422069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.422354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.422909 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394/users/291c9818263642f080c993ec53610ebc/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.424822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] RBAC: Authorizing `identity:create_grant(project_id=110958db305c4f80ae34050770c56394, user_id=291c9818263642f080c993ec53610ebc, role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.426777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.439263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 43/81] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:44 2022] PUT /identity/v3/projects/110958db305c4f80ae34050770c56394/users/291c9818263642f080c993ec53610ebc/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:28:44.440206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.519194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.524126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1699-5037-463e-a40d-79f43935ff58 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.562577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.563830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa78a1-79cb-415a-9d4b-3e5272e1080c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.564431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.564856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.565304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] PATH_INFO: `/v3/projects/39741be1d8c64f1db84f47ffbd6354fe` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.566824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] RBAC: Authorizing `identity:get_project(project_id=39741be1d8c64f1db84f47ffbd6354fe)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.568329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.570330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 39/82] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:44 2022] GET /identity/v3/projects/39741be1d8c64f1db84f47ffbd6354fe => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:44.570827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.594591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.609731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.610353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.610750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.611030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.611836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.613367 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.649596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 44/83] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:28:44 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:44.651538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.685633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504134df-6c75-40cb-81e1-1251aa5f705b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.738819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.739644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.740084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.740487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394/users/291c9818263642f080c993ec53610ebc/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.742888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=110958db305c4f80ae34050770c56394, user_id=291c9818263642f080c993ec53610ebc, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.744366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.770349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 40/84] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:44 2022] PUT /identity/v3/projects/110958db305c4f80ae34050770c56394/users/291c9818263642f080c993ec53610ebc/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:28:44.771635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:44.809908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b344cd-cf99-48e1-b742-f5bf37b563fa None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.813483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:44.855195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944afa47-7978-4280-a07d-69ea2796c6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.856259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944afa47-7978-4280-a07d-69ea2796c6a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.856745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944afa47-7978-4280-a07d-69ea2796c6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.857991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944afa47-7978-4280-a07d-69ea2796c6a6 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394/users/cdd55bd4e43346b29310a317dad7a19b/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.860342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944afa47-7978-4280-a07d-69ea2796c6a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=110958db305c4f80ae34050770c56394, user_id=cdd55bd4e43346b29310a317dad7a19b, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.862138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944afa47-7978-4280-a07d-69ea2796c6a6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.887124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 45/85] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:44 2022] PUT /identity/v3/projects/110958db305c4f80ae34050770c56394/users/cdd55bd4e43346b29310a317dad7a19b/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:28:44.913050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7288-f090-4c1f-9436-e2843fe582f9 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:44.966723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d4b52-0aad-4921-9827-964ad0d8cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:44.967884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d4b52-0aad-4921-9827-964ad0d8cbf5 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:44.968382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d4b52-0aad-4921-9827-964ad0d8cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:44.968961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d4b52-0aad-4921-9827-964ad0d8cbf5 None admin] PATH_INFO: `/v3/projects/39741be1d8c64f1db84f47ffbd6354fe/users/291c9818263642f080c993ec53610ebc/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:44.973418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d4b52-0aad-4921-9827-964ad0d8cbf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=39741be1d8c64f1db84f47ffbd6354fe, user_id=291c9818263642f080c993ec53610ebc, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:44.975392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d4b52-0aad-4921-9827-964ad0d8cbf5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:44.994575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 41/86] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:44 2022] PUT /identity/v3/projects/39741be1d8c64f1db84f47ffbd6354fe/users/291c9818263642f080c993ec53610ebc/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:28:47.351683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3c4f0-f7e6-4a7c-bf90-30e9719fe5bf None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:47.352680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3c4f0-f7e6-4a7c-bf90-30e9719fe5bf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:47.353163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3c4f0-f7e6-4a7c-bf90-30e9719fe5bf None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:47.354737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 46/87] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:47.360091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f59ea-2fb6-45bc-a7f6-fd06eb1f8437 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:47.360884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f59ea-2fb6-45bc-a7f6-fd06eb1f8437 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:47.361349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f59ea-2fb6-45bc-a7f6-fd06eb1f8437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:47.388615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-cf4f59ea-2fb6-45bc-a7f6-fd06eb1f8437 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:47.390126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cf4f59ea-2fb6-45bc-a7f6-fd06eb1f8437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:47.497741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 42/88] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:47.575561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0840b-5f29-4a68-995b-c7d55a2dbd5b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:47.575887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0840b-5f29-4a68-995b-c7d55a2dbd5b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:47.576109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0840b-5f29-4a68-995b-c7d55a2dbd5b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:47.577511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 47/89] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:47.583200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63823df7-227d-4aaf-8a44-4550794f7816 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:47.583520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63823df7-227d-4aaf-8a44-4550794f7816 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:47.583801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63823df7-227d-4aaf-8a44-4550794f7816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:47.606540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-63823df7-227d-4aaf-8a44-4550794f7816 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:47.607670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-63823df7-227d-4aaf-8a44-4550794f7816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:47.619115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:47.700955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 43/90] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:47.703211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977523-57ee-43af-9c74-654e75e0efb2 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:47.703531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977523-57ee-43af-9c74-654e75e0efb2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:47.703807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977523-57ee-43af-9c74-654e75e0efb2 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:47.705137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 48/91] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:47.720689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:47.720977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:47.721257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:47.744568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:47.746055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:47.831560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 44/92] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:48.004441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977523-57ee-43af-9c74-654e75e0efb2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.061158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.124756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.148185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977523-57ee-43af-9c74-654e75e0efb2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.209522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d102e-6e9e-4257-9f6a-a33ca30a02b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.210164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d102e-6e9e-4257-9f6a-a33ca30a02b4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.210447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d102e-6e9e-4257-9f6a-a33ca30a02b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.210719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d102e-6e9e-4257-9f6a-a33ca30a02b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.225161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.239889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d102e-6e9e-4257-9f6a-a33ca30a02b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-558d102e-6e9e-4257-9f6a-a33ca30a02b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:48.247451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:48.248292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 49/93] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:48.249008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c1f9-8d4a-4d9d-b920-40f3bc7cb889 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.249738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c1f9-8d4a-4d9d-b920-40f3bc7cb889 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.249738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c1f9-8d4a-4d9d-b920-40f3bc7cb889 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.249845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 50/94] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:48.255155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.255389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.255665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.273372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:48.273744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d4ca8-b5ad-4434-b6ca-cf382e815f1c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.274567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:48.316306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.317574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.317574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.317574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.328587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:48.333502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:48.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:48.335021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 45/95] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:28:48.335642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.356324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 51/96] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:48.360626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.401458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.451065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe2ce-6b5d-4fb5-b8f8-e4aafb492519 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.495860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.496480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.496758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.497028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.497702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:48.498005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:48.499369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b126c8-eae7-4052-884b-361ecd9bf9a1 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:48.502670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.510260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60630f6a-81c4-4438-b098-97c42d657826 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.515736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 46/97] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:48.517185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3fa77-b22d-4790-a9b8-db68c25fd35e None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.517743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3fa77-b22d-4790-a9b8-db68c25fd35e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.518248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3fa77-b22d-4790-a9b8-db68c25fd35e None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.520328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 47/98] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:48.524883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3fa77-b22d-4790-a9b8-db68c25fd35e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.552900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.553541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.553818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.554136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.564670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:48.569844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:48.570662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:48.571398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 52/99] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:48.572035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.663696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3fa77-b22d-4790-a9b8-db68c25fd35e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.667183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.678154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5d1a-dea4-4eb8-9062-c57e6d0d6261 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.700609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.701536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.701818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.702091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.702766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] RBAC: Adding query filter params (name=member) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:48.703160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:48.704565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2e489-f8c2-4121-94c1-9c728fbfbb4b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:48.714253 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 48/100] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:48.715619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75902-e744-4ae4-934a-a6290165569a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.715896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75902-e744-4ae4-934a-a6290165569a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.716169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75902-e744-4ae4-934a-a6290165569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.720495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.721130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.721412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.721686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] PATH_INFO: `/v3/users/291c9818263642f080c993ec53610ebc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.722987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] RBAC: Authorizing `identity:get_user(user_id=291c9818263642f080c993ec53610ebc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:48.724477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:48.726181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 53/101] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/users/291c9818263642f080c993ec53610ebc => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:48.730759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.736989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2df75902-e744-4ae4-934a-a6290165569a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:48.738707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2df75902-e744-4ae4-934a-a6290165569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:48.802917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.816701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 49/102] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:48.823960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df75902-e744-4ae4-934a-a6290165569a None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.853128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d49b1-5062-4cf4-b4d0-357216e3c305 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.892693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.893303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.893673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.893885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.894550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:48.894847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:48.896528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:48.905245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 54/103] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:48.905245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:48.927325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df75902-e744-4ae4-934a-a6290165569a None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:48.930078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:48.962212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:48.962847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:48.963129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:48.963406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:48.975228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:48.980064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:48.981084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:48.982341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 50/104] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:48.986547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.017214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b39a6-72c4-452e-8f63-80a4e2ba4116 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.038379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.053488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c2f8-2c8d-4af9-a8b9-f5d9eb6efa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.054359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c2f8-2c8d-4af9-a8b9-f5d9eb6efa26 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.054830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c2f8-2c8d-4af9-a8b9-f5d9eb6efa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.055311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c2f8-2c8d-4af9-a8b9-f5d9eb6efa26 None admin] PATH_INFO: `/v3/users/291c9818263642f080c993ec53610ebc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.056917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c2f8-2c8d-4af9-a8b9-f5d9eb6efa26 None admin] RBAC: Authorizing `identity:get_user(user_id=291c9818263642f080c993ec53610ebc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.058560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c2f8-2c8d-4af9-a8b9-f5d9eb6efa26 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.060595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 55/105] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/users/291c9818263642f080c993ec53610ebc => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.062010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bd074-64fc-404d-bc5d-4fbc1be63409 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.062491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bd074-64fc-404d-bc5d-4fbc1be63409 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.063026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bd074-64fc-404d-bc5d-4fbc1be63409 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.064471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 56/106] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:49.065802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bd074-64fc-404d-bc5d-4fbc1be63409 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.093747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee8278-b7df-49e6-9d08-c65f6db0238e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.128462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.129069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.129346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.129622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.155428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.169535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bd074-64fc-404d-bc5d-4fbc1be63409 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.197668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.199128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.202044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.202670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.202945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.203213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] PATH_INFO: `/v3/users/bd4619ddd49847b4bef1ed107407e1ad` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.216481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=bd4619ddd49847b4bef1ed107407e1ad)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.217959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.219582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 57/107] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/users/bd4619ddd49847b4bef1ed107407e1ad => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.220339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.247946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.297888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 51/108] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:28:49.299106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.321825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941131e-b252-4e52-837f-78f8bcd38fd6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.355676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.356284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.356566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.357088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.357493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:49.357793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.359179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91290e82-4e29-4f27-851c-640ca116358c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.368311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 58/109] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.368311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.368311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.368311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.388339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:49.389678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:49.391626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.395243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06bca-5fa7-4ba7-b7a0-8de56bbcf44c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.429743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.430357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.430637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.430914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.449164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.461998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 59/110] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:49.462849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.509958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.511449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.571091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e5f21-783b-49d5-9b96-bd78816fb947 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.606346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.606986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.607286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.607605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.614748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 52/111] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:28:49.615450 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.619782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.624111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:49.630182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:49.630988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:49.631803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:49.635247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 60/112] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:49.635858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.678274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.732401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e56dd6-7a42-4337-9de7-0e91f001fc12 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.740720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07064bd1-0524-4801-83ef-950afe786365 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.765599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.766235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.766521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.766795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.767953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.769391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.771018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 53/113] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.771816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.776222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.776854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.777136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.777419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.778551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.780036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.781805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 61/114] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.782694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.846014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.870992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b82c5e-355d-4fa1-b089-3bcb176dcaf0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.880804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef11aa4-7d30-477f-b2e5-c666f1307492 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:49.887254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:49.903870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.904572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.904855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.905126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] PATH_INFO: `/v3/users/291c9818263642f080c993ec53610ebc` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.906243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] RBAC: Authorizing `identity:get_user(user_id=291c9818263642f080c993ec53610ebc)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.907955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.910049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 54/115] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/users/291c9818263642f080c993ec53610ebc => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.910883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:49.915755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:49.916504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:49.916907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:49.917287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:49.918508 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:49.919994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:49.921625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 62/116] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:49.922378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.012239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e25ab-0548-4a39-8dc6-6b0c7daab0be None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.017897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76047-a3ee-4655-babf-0b7bc4e9507f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.046002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.046763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.047334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.047680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.048507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:50.048817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.050786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.052226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.052849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.053132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.053406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.054528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.056002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.057641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 63/117] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:50.059188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.063881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 55/118] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:50.063881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.066866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.107240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.156114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa57687-3094-4e0d-9611-ca637410f3a0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.163994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175a15-0499-4b0c-bc31-312f81665dd6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.190196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.190822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.191101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.191379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.192136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] RBAC: Adding query filter params (role.id=ade080f5f1cd46b89ad7b9088d1f1654, scope.project.id=110958db305c4f80ae34050770c56394, user.id=291c9818263642f080c993ec53610ebc) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:50.192449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.193882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.197005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.197661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.197943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.198217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394/users/bd4619ddd49847b4bef1ed107407e1ad/roles/81fbc573f2db4065af353a8879e07c88` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.199647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 56/119] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/role_assignments?user.id=291c9818263642f080c993ec53610ebc&scope.project.id=110958db305c4f80ae34050770c56394&role.id=ade080f5f1cd46b89ad7b9088d1f1654 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:50.200142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] RBAC: Authorizing `identity:create_grant(project_id=110958db305c4f80ae34050770c56394, user_id=bd4619ddd49847b4bef1ed107407e1ad, role_id=81fbc573f2db4065af353a8879e07c88)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.200526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.201708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.215659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 64/120] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:50 2022] PUT /identity/v3/projects/110958db305c4f80ae34050770c56394/users/bd4619ddd49847b4bef1ed107407e1ad/roles/81fbc573f2db4065af353a8879e07c88 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:28:50.216523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.292774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.342745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813aeaf4-78ad-4d11-80be-3075f51d872c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.349209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b759f-385d-4e52-a720-17b69e89b212 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.367834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.390605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.391393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.391804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.392203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.393069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] RBAC: Adding query filter params (role.id=2985373b41e749088f96f5c6c9b092c8, scope.project.id=110958db305c4f80ae34050770c56394, user.id=291c9818263642f080c993ec53610ebc) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:50.393488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.395729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.399559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.400204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.401323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.401685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.410313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 65/121] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/role_assignments?user.id=291c9818263642f080c993ec53610ebc&scope.project.id=110958db305c4f80ae34050770c56394&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:50.416620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.491814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.493328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.575949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6dd3b-dd97-4194-95c4-a696cce80601 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.606445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.617512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.618235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.618750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.618941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.619605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.621139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.633737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 57/122] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:28:50.635127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.653248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 66/123] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:28:50 2022] POST /identity/v3/projects => generated 302 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:50.654327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.666511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.784574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d0aeb-0dcd-46d7-a409-4bbcb27f9a87 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.794700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d08c4-a9b4-4ece-a915-6cec806db0fe None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.832511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.833462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.833811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.834155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] PATH_INFO: `/v3/users/cdd55bd4e43346b29310a317dad7a19b` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.835459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] RBAC: Authorizing `identity:get_user(user_id=cdd55bd4e43346b29310a317dad7a19b)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.837291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.839299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 58/124] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/users/cdd55bd4e43346b29310a317dad7a19b => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:50.839792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:50.840700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:50.841147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:50.841606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] PATH_INFO: `/v3/projects/39741be1d8c64f1db84f47ffbd6354fe` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:50.844338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] RBAC: Authorizing `identity:get_project(project_id=39741be1d8c64f1db84f47ffbd6354fe)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:50.844899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.846412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:50.848616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 67/125] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/projects/39741be1d8c64f1db84f47ffbd6354fe => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:50.856961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:50.922770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.952682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:50.978849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00fd4f-d14f-4358-af5b-afdcc9b441fe None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:50.991944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290af3ab-010e-4689-833d-bb5866375daf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:51.026497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:51.027179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:51.027500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:51.027783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:51.028776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd373-edde-497b-94c7-443237384865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:51.029532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd373-edde-497b-94c7-443237384865 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:51.029965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd373-edde-497b-94c7-443237384865 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:51.030377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd373-edde-497b-94c7-443237384865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:51.031239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd373-edde-497b-94c7-443237384865 None admin] RBAC: Adding query filter params (role.id=2985373b41e749088f96f5c6c9b092c8, scope.project.id=39741be1d8c64f1db84f47ffbd6354fe, user.id=291c9818263642f080c993ec53610ebc) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:51.031739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd373-edde-497b-94c7-443237384865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:51.033627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd373-edde-497b-94c7-443237384865 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:51.040058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 68/126] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/role_assignments?user.id=291c9818263642f080c993ec53610ebc&scope.project.id=39741be1d8c64f1db84f47ffbd6354fe&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:51.113673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:51.115198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:51.207465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:51.234069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 59/127] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 06:28:51.240245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd373-edde-497b-94c7-443237384865 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:51.345897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd373-edde-497b-94c7-443237384865 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:51.357085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:51.381544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94090fb6-3e3f-49f2-a0c3-0db5960f9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:51.382248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090fb6-3e3f-49f2-a0c3-0db5960f9aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:51.382603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090fb6-3e3f-49f2-a0c3-0db5960f9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:51.382951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090fb6-3e3f-49f2-a0c3-0db5960f9aa7 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:51.384177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090fb6-3e3f-49f2-a0c3-0db5960f9aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:51.385688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090fb6-3e3f-49f2-a0c3-0db5960f9aa7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:51.387692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 69/128] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:51 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:51.393441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:51.498691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84f4c8-a769-4fb5-8899-89ab1978dbe8 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:51.531239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:51.531869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:51.532407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:51.532678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:51.533397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=110958db305c4f80ae34050770c56394, user.id=cdd55bd4e43346b29310a317dad7a19b) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:51.533701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:51.535724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1407-064f-4b02-ac92-b5344949caee None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:51.541466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 60/129] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:28:51 2022] GET /identity/v3/role_assignments?user.id=cdd55bd4e43346b29310a317dad7a19b&scope.project.id=110958db305c4f80ae34050770c56394&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:52.571955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbed455-b501-4fcc-a4e9-0b908a011f6b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:52.572561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbed455-b501-4fcc-a4e9-0b908a011f6b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:52.572561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbed455-b501-4fcc-a4e9-0b908a011f6b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:52.573861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 70/130] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:52.579227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49784a-842f-46da-8443-2da98f60bf2a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:52.579657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49784a-842f-46da-8443-2da98f60bf2a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:52.579941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49784a-842f-46da-8443-2da98f60bf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:52.598915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-7c49784a-842f-46da-8443-2da98f60bf2a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:52.600132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7c49784a-842f-46da-8443-2da98f60bf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:52.637929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:52.665340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 61/131] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:52.924492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18725f-0201-44aa-99b2-3c69a67a11c1 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:52.924732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18725f-0201-44aa-99b2-3c69a67a11c1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:52.925026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18725f-0201-44aa-99b2-3c69a67a11c1 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:52.926540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 71/132] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:52.931544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:52.931836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:52.932115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:52.935847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18725f-0201-44aa-99b2-3c69a67a11c1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:52.950530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:52.951757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:53.016150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 62/133] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:53.035461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18725f-0201-44aa-99b2-3c69a67a11c1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.067719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.068345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.068624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.068896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.078491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:53.084389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:53.085356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:53.086168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 72/134] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:28:53.091388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.186554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fab53-cabb-437b-9923-e82c56e86d9c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.189001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:53.221215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.221850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.222374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.222657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.223329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:53.223665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.225032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.233166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 63/135] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:53.239360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.277520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:53.289762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.337121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51e4d-bfd9-46a0-a51b-b5ccdf547179 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.369466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.370087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.370364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.370628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] PATH_INFO: `/v3/users/bd4619ddd49847b4bef1ed107407e1ad` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.371770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] RBAC: Authorizing `identity:get_user(user_id=bd4619ddd49847b4bef1ed107407e1ad)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.373206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.374779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 73/136] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:53 2022] GET /identity/v3/users/bd4619ddd49847b4bef1ed107407e1ad => generated 289 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:53.380778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.387994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cb931-f177-408e-bd9d-34ec4d60b283 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.423965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.424599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.424883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.425699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.428027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.429464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.431023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 64/137] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:53.436350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.469523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:53.483328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c2c0f-d122-4bff-8bbf-3479e3727b01 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.490920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:53.517813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.518487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.518797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.519067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.535642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75b249-08fe-429e-ab24-0d4500e24c5d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.570377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.570978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.571267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.571581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.572233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.573668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.584350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.586262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.595232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 65/138] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:28:53 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:28:53.691079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 74/139] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 06:28:53.696971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.789743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:53.799910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2de4f-360a-4275-ad61-5e9d6c8a01cf None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.834534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7603a22-8e09-4d8c-9e92-50f5f5430a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.835169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603a22-8e09-4d8c-9e92-50f5f5430a73 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.835481 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603a22-8e09-4d8c-9e92-50f5f5430a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.835759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7603a22-8e09-4d8c-9e92-50f5f5430a73 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.836876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7603a22-8e09-4d8c-9e92-50f5f5430a73 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.838305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7603a22-8e09-4d8c-9e92-50f5f5430a73 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.839966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 66/140] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:53 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:53.846120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:53.854956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:53.950202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c920-27c2-4767-bcc0-f68edef13ea4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:53.988379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:53.988985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:53.989256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:53.989524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:53.990251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] RBAC: Adding query filter params (role.id=81fbc573f2db4065af353a8879e07c88, scope.project.id=110958db305c4f80ae34050770c56394, user.id=bd4619ddd49847b4bef1ed107407e1ad) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:53.990551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:53.992039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5518d-dca8-4358-b981-ff13a10f8c25 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:53.998452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 75/141] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:28:53 2022] GET /identity/v3/role_assignments?user.id=bd4619ddd49847b4bef1ed107407e1ad&scope.project.id=110958db305c4f80ae34050770c56394&role.id=81fbc573f2db4065af353a8879e07c88 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:55.848142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ee498-2da3-4af6-b9e0-48514dd3ce35 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:55.848619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ee498-2da3-4af6-b9e0-48514dd3ce35 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:55.848619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ee498-2da3-4af6-b9e0-48514dd3ce35 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:55.849953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 67/142] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:55.855271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade074e-6b9f-4e47-b03b-49dad7d793a6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:55.855696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade074e-6b9f-4e47-b03b-49dad7d793a6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:55.855861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade074e-6b9f-4e47-b03b-49dad7d793a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:55.875579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1ade074e-6b9f-4e47-b03b-49dad7d793a6 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:55.876821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1ade074e-6b9f-4e47-b03b-49dad7d793a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:55.945265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:55.950705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 76/143] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:55.986916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6abcb0-e90e-479f-96e0-fa9e3fbae40c None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:55.987222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6abcb0-e90e-479f-96e0-fa9e3fbae40c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:55.987593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6abcb0-e90e-479f-96e0-fa9e3fbae40c None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:55.989585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 68/144] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:55.994411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:55.994702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:55.994976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.013066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:56.014233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:56.055891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a2c25-0c48-4a73-88ee-8231a429ad7b None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.056189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a2c25-0c48-4a73-88ee-8231a429ad7b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.056457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a2c25-0c48-4a73-88ee-8231a429ad7b None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.057958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 69/145] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:28:56.062594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.062884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.063168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.082319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:56.087612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 77/146] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:56.087758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:56.155771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 70/147] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:56.221204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.322751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a8ce1-1289-4951-8aa3-ca4968ed6e7e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.352162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:56.358683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.359315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.359630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.359904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.365633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.370139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:56.375025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:56.375858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:56.376765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 78/148] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:56.381786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.463384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:56.474951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845e06d-7173-42e3-b409-b37b8071765c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.490140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf119534-153e-42a1-9397-3abaf39cc701 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.507562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.508192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.508474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.508753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.518637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:56.523252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.523879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:56.524052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:56.524989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:28:56.525739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:28:56.526308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.526308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.526308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:56.526564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 71/149] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:28:56.526634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.526702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.527370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.535680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 79/150] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:56.536526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.588836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:56.626373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f5b47-1885-4191-848c-05f0ce365aba None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.636023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab33ab-b9c3-43eb-b3c2-a318ccfb26f7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.660498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.661135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.661415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.662801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.664280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.666012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 72/151] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:56.666613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.669144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.669753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.670035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.670520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.670970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:56.671273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.672718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.680637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 80/152] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:56.681396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.688970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:56.769945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487e74-1b0b-4c88-a974-b0d45a76591a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.779290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e710a83-236a-473f-abb3-65e69e6201d0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.792517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:56.802802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.803465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.803747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.804021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] PATH_INFO: `/v3/users/a7b746786afe45428b394f1b40727f42` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.812436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.813031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.813303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.813568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.814204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.816188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.817255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b746786afe45428b394f1b40727f42)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.818682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.820355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 73/153] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/users/a7b746786afe45428b394f1b40727f42 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:56.821120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.839138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 81/154] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:28:56 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:28:56.840006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:56.915735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:56.951292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d961c-f474-436e-bef5-29125889cdc2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.953821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875a68b-b9d6-4257-a46c-dad1bcccbd92 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:56.991755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.992408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:56.992532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.992713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.992955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.993053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:56.993313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:56.993579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] PATH_INFO: `/v3/users/a7b746786afe45428b394f1b40727f42` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:56.994170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.994719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b746786afe45428b394f1b40727f42)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:56.995683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.996194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:56.997580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 82/155] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:56.997773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 74/156] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:28:56 2022] GET /identity/v3/users/a7b746786afe45428b394f1b40727f42 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:57.003803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:57.004002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:57.049404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:57.126529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c57b5-1b02-455c-b820-fad6b66dfd50 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:57.134964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bc2dc-8ed6-43f1-ba6b-d989663c4417 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:57.166036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:57.166731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:57.167013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:57.167316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:57.168420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:57.169876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:57.171702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 83/157] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:28:57 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:57.173425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9ef3e-e4f2-4298-bbe4-f49fc35b6702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:57.174085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9ef3e-e4f2-4298-bbe4-f49fc35b6702 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:57.174364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9ef3e-e4f2-4298-bbe4-f49fc35b6702 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:57.174625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9ef3e-e4f2-4298-bbe4-f49fc35b6702 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/a7b746786afe45428b394f1b40727f42/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:57.177418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9ef3e-e4f2-4298-bbe4-f49fc35b6702 None admin] RBAC: Authorizing `identity:create_grant(project_id=4712eac5a11d44b7aa936726f9d6d34c, user_id=a7b746786afe45428b394f1b40727f42, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:57.178045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:57.178849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9ef3e-e4f2-4298-bbe4-f49fc35b6702 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:57.195054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 75/158] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:57 2022] PUT /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/a7b746786afe45428b394f1b40727f42/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:28:57.294209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd9e6-ef53-4a23-a19a-7ddff44ce476 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:57.302840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:57.332717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a2c1-a62f-4a1f-84be-8b465aa56f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:57.333476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a2c1-a62f-4a1f-84be-8b465aa56f1a None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:57.333761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a2c1-a62f-4a1f-84be-8b465aa56f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:57.334038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a2c1-a62f-4a1f-84be-8b465aa56f1a None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/a7b746786afe45428b394f1b40727f42/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:57.336494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9a2c1-a62f-4a1f-84be-8b465aa56f1a None admin] RBAC: Authorizing `identity:create_grant(project_id=4712eac5a11d44b7aa936726f9d6d34c, user_id=a7b746786afe45428b394f1b40727f42, role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:57.338117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9a2c1-a62f-4a1f-84be-8b465aa56f1a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:57.353875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 84/159] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:28:57 2022] PUT /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/a7b746786afe45428b394f1b40727f42/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:28:59.220097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cc336-67a5-43f5-8f1d-a84855fdec92 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.220918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cc336-67a5-43f5-8f1d-a84855fdec92 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.220918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cc336-67a5-43f5-8f1d-a84855fdec92 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.222639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 76/160] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:28:59.228000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ff60e-5616-4435-8cf6-61547c67e5e8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.228626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ff60e-5616-4435-8cf6-61547c67e5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.228626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ff60e-5616-4435-8cf6-61547c67e5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.247940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d39ff60e-5616-4435-8cf6-61547c67e5e8 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:59.249107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d39ff60e-5616-4435-8cf6-61547c67e5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:59.287678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636074a2-313a-4855-96f8-9501ee56a88f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.288066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636074a2-313a-4855-96f8-9501ee56a88f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.288328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636074a2-313a-4855-96f8-9501ee56a88f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.289902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 77/161] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:59.297923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fffad0-52dc-4ab4-a2c1-71314fda4a15 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.298764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fffad0-52dc-4ab4-a2c1-71314fda4a15 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.299151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fffad0-52dc-4ab4-a2c1-71314fda4a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.307375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:59.324035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d9fffad0-52dc-4ab4-a2c1-71314fda4a15 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:59.325769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d9fffad0-52dc-4ab4-a2c1-71314fda4a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:59.332754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 85/162] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:28:59.398896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 78/163] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:59.467804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e72431-1d7e-4ee2-8e5f-96a09c24b931 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.468235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e72431-1d7e-4ee2-8e5f-96a09c24b931 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.468608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e72431-1d7e-4ee2-8e5f-96a09c24b931 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.470363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 86/164] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:59.476846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.477243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.477613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.495892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:59.497060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:59.566471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 79/165] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:59.610535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e72431-1d7e-4ee2-8e5f-96a09c24b931 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:59.670593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:59.731683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:59.743139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:59.744040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e72431-1d7e-4ee2-8e5f-96a09c24b931 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:59.784793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d71ff-80fe-4e28-849e-012f4194abf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:59.785031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbe36-d53a-4289-8584-719298c72f75 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:59.785421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d71ff-80fe-4e28-849e-012f4194abf2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.785694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d71ff-80fe-4e28-849e-012f4194abf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.785962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d71ff-80fe-4e28-849e-012f4194abf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.796061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d71ff-80fe-4e28-849e-012f4194abf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-592d71ff-80fe-4e28-849e-012f4194abf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:28:59.800929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:28:59.801755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:28:59.802592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 87/166] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:28:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:28:59.803212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687a6c-57f3-4614-9da5-524b6e0567aa None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.803770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687a6c-57f3-4614-9da5-524b6e0567aa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.803770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687a6c-57f3-4614-9da5-524b6e0567aa None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.805046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 88/167] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:28:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:28:59.809614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687a6c-57f3-4614-9da5-524b6e0567aa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:59.825982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ce85a-9da6-4c19-8906-c3be66c40046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:59.826985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce85a-9da6-4c19-8906-c3be66c40046 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.827410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce85a-9da6-4c19-8906-c3be66c40046 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.827707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce85a-9da6-4c19-8906-c3be66c40046 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.828794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ce85a-9da6-4c19-8906-c3be66c40046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:59.830216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ce85a-9da6-4c19-8906-c3be66c40046 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:59.832483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 80/168] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:28:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:28:59.833560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.833843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.834111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.852415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:28:59.853580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:28:59.911950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687a6c-57f3-4614-9da5-524b6e0567aa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:28:59.918685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 81/169] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:28:59.919592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:59.948323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:28:59.949419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:28:59.949767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:28:59.950038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:28:59.950728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:28:59.951026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:28:59.952546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:28:59.961480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 89/170] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:28:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:28:59.962110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:28:59.980402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:28:59.984194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.026097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1332ebc-7689-435a-ad64-5e390d603a28 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.062668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.063504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.063952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.064363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.064950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af9801-e62e-4840-a1fc-c57ff0399aa8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.065310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.067188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.094059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 82/171] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:28:59 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:00.100557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.101730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.102576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.102882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.103273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.118618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:00.123996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:00.124987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:00.125795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 90/172] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:00.132197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.220414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.241345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bd371-6046-4bec-ab1c-aedd88aaadf9 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.247022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf409f6-8276-42e1-a586-eae987335bb3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.259399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.282545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.283526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.283991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.284882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] PATH_INFO: `/v3/users/50967335a770417e858f62de8e012a41` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.286989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.287624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.287899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.288185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.288856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:00.289153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.290531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.299582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] RBAC: Authorizing `identity:get_user(user_id=50967335a770417e858f62de8e012a41)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.301619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.301724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 91/173] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:00.302540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.303659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 83/174] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/users/50967335a770417e858f62de8e012a41 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:00.307805 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.420820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f52b-2eed-4288-8ecf-6ee58572ba9a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.436207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade39e9-e265-4cf3-a10a-2f952afe2805 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.469044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.469338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.469616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.469895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.477449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.478823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.478823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.478993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] PATH_INFO: `/v3/users/a7b746786afe45428b394f1b40727f42` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.488082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] RBAC: Authorizing `identity:get_user(user_id=a7b746786afe45428b394f1b40727f42)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.488262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.488354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.488354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 84/175] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/users/a7b746786afe45428b394f1b40727f42 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:00.488354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.488527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:00.492401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:00.493253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:00.494005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 92/176] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:00.494637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.542633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.610479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeda89-6b66-4de6-9b44-62529d02bb3c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.635724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844ba17-0a12-46f8-968f-958de8c43e5b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.649280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.650186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.650465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.650733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.651898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.653898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.656081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 85/177] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:00.661606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.680903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.681510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.681784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.682051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.761184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.765834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.768413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.791657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dbf03-a24f-47c5-9feb-cc07ea991e20 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.805378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.833619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:00.834514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:00.835223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:00.835690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:00.837284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:00.837764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:00.839391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:00.849615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 86/178] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:00.850596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.882242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 93/179] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:00.888858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:00.975103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add332-5e4f-4c7d-a764-bd7355e19bcc None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:00.993302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:00.997213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279e746-73f7-4698-9831-e03f1df7aa07 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:01.017314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.018278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.019303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.019767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/50967335a770417e858f62de8e012a41/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.021970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] RBAC: Authorizing `identity:create_grant(project_id=4712eac5a11d44b7aa936726f9d6d34c, user_id=50967335a770417e858f62de8e012a41, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.023935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.037215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.037924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.038299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.038668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] PATH_INFO: `/v3/users/a7b746786afe45428b394f1b40727f42` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.039888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] RBAC: Authorizing `identity:get_user(user_id=a7b746786afe45428b394f1b40727f42)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.041249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 87/180] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:00 2022] PUT /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/50967335a770417e858f62de8e012a41/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:01.047541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.047541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 94/181] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:00 2022] GET /identity/v3/users/a7b746786afe45428b394f1b40727f42 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:01.048648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:01.050999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:01.093925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:01.182539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb217e5-7678-4f42-85e7-563b0bbc151d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:01.223703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.224716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.225226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.225697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.280702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb2abb-58ee-445b-aa13-b8a333cb4316 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:01.308871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.310942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.322091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.323259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.323695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.323972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.325758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.327239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.329526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 95/182] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:01 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:01.336401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:01.349642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:01.380163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:01.453294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 88/183] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:01.458657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:01.506486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce93b-1e56-456e-927e-943446259ab9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:01.591067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.591691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.591983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.592255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.592997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=4712eac5a11d44b7aa936726f9d6d34c, user.id=a7b746786afe45428b394f1b40727f42) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:01.593303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.594764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.602604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2733da-969d-4fdc-a505-3bc1ecbdc0ee None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:01.614129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 96/184] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:01 2022] GET /identity/v3/role_assignments?user.id=a7b746786afe45428b394f1b40727f42&scope.project.id=4712eac5a11d44b7aa936726f9d6d34c&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:01.636764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:01.643747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde96e9e-ec4c-4c47-92e6-5da813403a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.644408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde96e9e-ec4c-4c47-92e6-5da813403a61 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.644688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde96e9e-ec4c-4c47-92e6-5da813403a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.644962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde96e9e-ec4c-4c47-92e6-5da813403a61 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.646247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde96e9e-ec4c-4c47-92e6-5da813403a61 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.647764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde96e9e-ec4c-4c47-92e6-5da813403a61 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.649639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 89/185] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:01 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:01.655618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:01.808138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:01.811152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b874b-1a58-42fa-8244-e6278f30fb0c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:01.847651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:01.848177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:01.848449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:01.848711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:01.849436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] RBAC: Adding query filter params (role.id=ade080f5f1cd46b89ad7b9088d1f1654, scope.project.id=4712eac5a11d44b7aa936726f9d6d34c, user.id=a7b746786afe45428b394f1b40727f42) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:01.849737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:01.851168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d610f5-c1b5-4c02-ae19-7f3a5c8b740f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:01.857101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 97/186] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:01 2022] GET /identity/v3/role_assignments?user.id=a7b746786afe45428b394f1b40727f42&scope.project.id=4712eac5a11d44b7aa936726f9d6d34c&role.id=ade080f5f1cd46b89ad7b9088d1f1654 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:02.521832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a504bc-9dde-490d-b4aa-27ca8950fd28 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:02.522337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a504bc-9dde-490d-b4aa-27ca8950fd28 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:02.522415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a504bc-9dde-490d-b4aa-27ca8950fd28 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:02.523827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 90/187] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:02.528988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:02.529364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:02.529518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:02.548254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:02.549429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:02.578799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa25e1-b296-4c36-99ab-82b37587b2bc None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:02.579102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa25e1-b296-4c36-99ab-82b37587b2bc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:02.579355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa25e1-b296-4c36-99ab-82b37587b2bc None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:02.580874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 91/188] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:02.586402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:02.586622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:02.586900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:02.605645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:02.607151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:02.619455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 98/189] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:02.677905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 92/190] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:02.886903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:02.944152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:02.987995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:02.995242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f95c-da8d-4687-8b35-9db9ef70f67f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.029259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.029897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.030173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.030441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.031562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:03.033257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:03.035460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 99/191] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:03.040905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:03.055552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675acd1e-a6d2-4a6b-b574-ff2f18fcc2f7 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.087768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:03.095003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.095915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.096889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.097360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.108140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:03.113707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:03.114519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:03.115266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:03.117047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 93/192] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:03.122664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:03.152578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f620-fc61-4f6e-9bc6-4e8362b07736 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.195703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.197622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.198094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.198537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.199362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:03.200974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:03.235445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 100/193] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:03 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:29:03.235653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c090c-ce77-434b-9f6d-e9de1b592723 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.270145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.270771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.271051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.271322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.272032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:03.272336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:03.273746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:03.283866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 94/194] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:03.289914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:03.312863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:03.416936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f9ccf-4054-4e36-92f0-36bf8db9d7f6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.456146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571a4a-e5de-442a-9ba0-88ed598f1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.456735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571a4a-e5de-442a-9ba0-88ed598f1856 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.457015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571a4a-e5de-442a-9ba0-88ed598f1856 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.457290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571a4a-e5de-442a-9ba0-88ed598f1856 None admin] PATH_INFO: `/v3/users/119b062884914dfdae152d1e611edb41` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.472399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571a4a-e5de-442a-9ba0-88ed598f1856 None admin] RBAC: Authorizing `identity:get_user(user_id=119b062884914dfdae152d1e611edb41)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:03.473867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571a4a-e5de-442a-9ba0-88ed598f1856 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:03.475891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 101/195] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:03 2022] GET /identity/v3/users/119b062884914dfdae152d1e611edb41 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:03.481308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:03.499378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd68c0a-7331-411d-a696-6c1c854ccb0b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.499705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd68c0a-7331-411d-a696-6c1c854ccb0b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.499964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd68c0a-7331-411d-a696-6c1c854ccb0b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.501422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 102/196] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:03.507712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.508266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.508739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.529728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:03.531685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:03.543765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:03.606412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bda387-325c-4043-8a1d-e14cf4288d25 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.609316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 103/197] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:03.641037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.641727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.642009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.642338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.643511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:03.645528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:03.647479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 95/198] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:03 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:03.653138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:03.664685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:03.761090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023e3f6-242b-4be5-ac83-e2fbce92300a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:03.798694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:03.799775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:03.800279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:03.800747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/119b062884914dfdae152d1e611edb41/roles/81fbc573f2db4065af353a8879e07c88` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:03.802923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] RBAC: Authorizing `identity:create_grant(project_id=4712eac5a11d44b7aa936726f9d6d34c, user_id=119b062884914dfdae152d1e611edb41, role_id=81fbc573f2db4065af353a8879e07c88)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:03.804638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:03.820385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 104/199] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:03 2022] PUT /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/users/119b062884914dfdae152d1e611edb41/roles/81fbc573f2db4065af353a8879e07c88 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:03.907053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:04.026435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6f976-87a3-412e-b988-657f4a37b6c9 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:04.048287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:04.063333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:04.063999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:04.064283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:04.064558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:04.075863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:04.082480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:04.083543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:04.084303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 96/200] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:04.089817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:04.160375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:04.198458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b9a47-22a6-463b-b86a-a0416320fb26 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:04.234816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:04.235449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:04.235737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:04.236090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:04.236880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:04.237186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:04.238740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:04.248860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 105/201] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:04.255232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:04.360914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a86326-bd3f-4c7a-a033-8a638394aaf0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:04.398290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:04.399299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:04.399811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:04.400284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] PATH_INFO: `/v3/users/50967335a770417e858f62de8e012a41` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:04.408238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] RBAC: Authorizing `identity:get_user(user_id=50967335a770417e858f62de8e012a41)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:04.415211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:04.417200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 97/202] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:04 2022] GET /identity/v3/users/50967335a770417e858f62de8e012a41 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:04.422911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:04.531169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0e34d-e14f-4be2-8c95-e15f0b442fdd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:04.558343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:04.567772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:04.568384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:04.568662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:04.568933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:04.638736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:04.640152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:04.742064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 106/203] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:04.748158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:04.803596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:04.858538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee7c3-20f0-4b38-9c20-95171334a291 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:04.892483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f5d50-513e-42e9-b92c-20b5ca3da31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:04.893043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f5d50-513e-42e9-b92c-20b5ca3da31f None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:04.893324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f5d50-513e-42e9-b92c-20b5ca3da31f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:04.893643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f5d50-513e-42e9-b92c-20b5ca3da31f None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:04.895276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f5d50-513e-42e9-b92c-20b5ca3da31f None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:04.896752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f5d50-513e-42e9-b92c-20b5ca3da31f None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:04.898680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 98/204] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:04 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:04.904548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:04.933137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:05.005892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440901-3ead-4638-b6f8-f75db08ad52f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:05.041056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:05.041900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:05.042324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:05.042739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:05.043639 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] RBAC: Adding query filter params (role.id=2985373b41e749088f96f5c6c9b092c8, scope.project.id=4712eac5a11d44b7aa936726f9d6d34c, user.id=50967335a770417e858f62de8e012a41) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:05.044133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:05.045741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2420a-4bc2-4027-a746-dc279b833570 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:05.052091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 107/205] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:04 2022] GET /identity/v3/role_assignments?user.id=50967335a770417e858f62de8e012a41&scope.project.id=4712eac5a11d44b7aa936726f9d6d34c&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:05.526498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320d6fd-103e-4fda-a00c-eb4dd8d68531 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:05.526846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320d6fd-103e-4fda-a00c-eb4dd8d68531 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:05.527044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320d6fd-103e-4fda-a00c-eb4dd8d68531 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:05.528570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 99/206] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:05.533937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf231ca-e53d-4e4a-b2ad-d6eda17503d8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:05.534892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf231ca-e53d-4e4a-b2ad-d6eda17503d8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:05.535052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf231ca-e53d-4e4a-b2ad-d6eda17503d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:05.553183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-baf231ca-e53d-4e4a-b2ad-d6eda17503d8 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:05.554340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-baf231ca-e53d-4e4a-b2ad-d6eda17503d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:05.624742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 108/207] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:05.690724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a2017-3ab7-441c-9257-2f37366cd4db None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:05.690928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a2017-3ab7-441c-9257-2f37366cd4db None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:05.691372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a2017-3ab7-441c-9257-2f37366cd4db None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:05.692886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 100/208] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:05.697911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:05.698201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:05.698475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:05.717293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:05.718468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:05.736065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:05.788554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 109/209] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:05.890963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a2017-3ab7-441c-9257-2f37366cd4db None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:05.991730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a2017-3ab7-441c-9257-2f37366cd4db None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.007363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:06.027211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.027908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.028182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.028448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.041942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:06.047156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:06.048135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:06.048996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 101/210] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:06.055174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.055679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.157589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39cb14-03df-42e2-8879-dd89a87bf467 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.160745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ed7d4-5610-4f52-bb15-35966d1bb87e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.190957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0fc67-883a-42a6-993b-fb1b0e5a7974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.191621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0fc67-883a-42a6-993b-fb1b0e5a7974 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.191902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0fc67-883a-42a6-993b-fb1b0e5a7974 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.192177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0fc67-883a-42a6-993b-fb1b0e5a7974 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.193274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0fc67-883a-42a6-993b-fb1b0e5a7974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:06.193682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.194283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.194558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.194733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0fc67-883a-42a6-993b-fb1b0e5a7974 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:06.194818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.195498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] RBAC: Adding query filter params (name=member) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:06.195795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:06.196344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 110/211] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:06.197150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51870ce-b48a-4fff-ab2a-2ad55bf6fadb None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:06.197454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bda85-30b7-4236-845c-42655c4c0a85 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.197735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bda85-30b7-4236-845c-42655c4c0a85 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.198005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bda85-30b7-4236-845c-42655c4c0a85 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.199390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 111/212] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:06.203929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bda85-30b7-4236-845c-42655c4c0a85 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.210918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 102/213] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:06.210918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.210918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.210918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.225589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:06.226773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:06.229876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:06.255529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:06.292352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 103/214] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:06.296309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.308778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bda85-30b7-4236-845c-42655c4c0a85 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.343894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.344507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.344783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.345055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.345716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:06.347208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:06.368325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 112/215] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:06 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:06.402283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd2eb3-7ecf-49c4-a477-0d07aa36629f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.448279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.449241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.449689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.450152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] PATH_INFO: `/v3/users/8a29ee29b8f74f549477dc7208b24daf` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.465999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] RBAC: Authorizing `identity:get_user(user_id=8a29ee29b8f74f549477dc7208b24daf)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:06.467848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:06.469724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 104/216] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/users/8a29ee29b8f74f549477dc7208b24daf => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:06.475643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.558237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:06.559163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.585464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1de6d4-1a77-43dd-a0a6-be73662f2316 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.596904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:06.627213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.627848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.628691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.629014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.629750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] RBAC: Adding query filter params (role.id=2985373b41e749088f96f5c6c9b092c8, user.id=8a29ee29b8f74f549477dc7208b24daf, scope.system=all) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:06.632858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:06.632858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:06.638280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 113/217] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/role_assignments?user.id=8a29ee29b8f74f549477dc7208b24daf&scope.system=all&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:06.696045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f85e7b-55da-45ae-a97a-ac2166e4fb17 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.736949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.737640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.737922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.738193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.751012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:06.756238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:06.757152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:06.757898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 105/218] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:06.763284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:06.885560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1e5ed-379b-4a07-b34b-525710d44e52 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:06.925890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:06.926842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:06.927126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:06.927473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:06.928161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:06.928463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:06.929956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:06.939473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 114/219] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:06.945268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:07.040045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:07.056198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242964c-e95d-4422-9b26-5fc8f561e3da None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:07.092066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:07.093104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:07.093582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:07.094077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] PATH_INFO: `/v3/users/119b062884914dfdae152d1e611edb41` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:07.095444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=119b062884914dfdae152d1e611edb41)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:07.097128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:07.099225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 106/220] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:06 2022] GET /identity/v3/users/119b062884914dfdae152d1e611edb41 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:07.104457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:07.110006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:07.218841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed025ae-7c10-4850-b359-90bb33e1481a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:07.253761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:07.254377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:07.254653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:07.254923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:07.325074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:07.326550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:07.328417 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:07.437892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 115/221] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:07.444582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:07.545102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff90b4c-d6db-44cc-a2e5-8564900df7c2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:07.584289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd81bf-a8e9-4222-addf-5debc094b40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:07.585309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd81bf-a8e9-4222-addf-5debc094b40a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:07.585792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd81bf-a8e9-4222-addf-5debc094b40a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:07.586288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd81bf-a8e9-4222-addf-5debc094b40a None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:07.587729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd81bf-a8e9-4222-addf-5debc094b40a None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:07.589383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd81bf-a8e9-4222-addf-5debc094b40a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:07.591666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 107/222] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:07 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:07.602714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:07.698658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790cc9-95b9-4184-8369-e35faf4c850d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:07.700199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:07.734181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:07.734784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:07.735065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:07.735340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:07.736098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] RBAC: Adding query filter params (role.id=81fbc573f2db4065af353a8879e07c88, scope.project.id=4712eac5a11d44b7aa936726f9d6d34c, user.id=119b062884914dfdae152d1e611edb41) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:07.736405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:07.738411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ccca1-4139-4ac0-b2aa-85566d787672 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:07.744582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 116/223] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:07 2022] GET /identity/v3/role_assignments?user.id=119b062884914dfdae152d1e611edb41&scope.project.id=4712eac5a11d44b7aa936726f9d6d34c&role.id=81fbc573f2db4065af353a8879e07c88 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:08.735741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39377d48-98dd-46f1-b6f3-171f82b79113 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:08.736237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39377d48-98dd-46f1-b6f3-171f82b79113 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:08.736237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39377d48-98dd-46f1-b6f3-171f82b79113 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:08.738189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 108/224] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:08.744343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3c4e-eee4-4298-9615-3f711a9f4dfe None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:08.745273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3c4e-eee4-4298-9615-3f711a9f4dfe None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:08.745554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3c4e-eee4-4298-9615-3f711a9f4dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:08.765015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-772c3c4e-eee4-4298-9615-3f711a9f4dfe None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:08.766249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-772c3c4e-eee4-4298-9615-3f711a9f4dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:08.834873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca8b7e-76a2-4bbd-b0c1-ded26a476e8f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:08.835158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca8b7e-76a2-4bbd-b0c1-ded26a476e8f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:08.835435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca8b7e-76a2-4bbd-b0c1-ded26a476e8f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:08.836764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 117/225] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:08.837018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 109/226] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:08.842160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:08.842448 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:08.842724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:08.860884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:08.862055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:08.868034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e13e7-d949-421f-97a4-2f269eb12d72 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:08.868320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e13e7-d949-421f-97a4-2f269eb12d72 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:08.868591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e13e7-d949-421f-97a4-2f269eb12d72 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:08.869922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 110/227] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:08.874836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:08.875117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:08.875389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:08.892743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:08.894476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:08.911275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:08.938661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 118/228] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:08.965994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 111/229] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:09.107397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.133819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.205573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af62a5-2881-4eaa-ac2f-39731843d04a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.206895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.245434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.246070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.246347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.246615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.256930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:09.261797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:09.262646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:09.263400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 119/230] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:09.264050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.307276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f73f50-d4de-4cde-9b16-b90e68263630 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.340179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.340804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.341091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.341361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.342448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.343917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.345476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 112/231] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:09.348464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.348592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.364090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbd7a-24fa-4f59-b1c4-3221de37e232 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.369608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.398972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f808-793e-4010-9bac-247efa587879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.399632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f808-793e-4010-9bac-247efa587879 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.399912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f808-793e-4010-9bac-247efa587879 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.400179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f808-793e-4010-9bac-247efa587879 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.413833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f808-793e-4010-9bac-247efa587879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-d709f808-793e-4010-9bac-247efa587879 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:09.418708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:09.419597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:09.420364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 120/232] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:09.420987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f808-793e-4010-9bac-247efa587879 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.451600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7041e6-57ef-4e95-82b8-aff24680bec0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.485818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.486418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.486690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.486956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.487640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:09.487937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.489309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.501619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 113/233] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:09.501619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.518152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f808-793e-4010-9bac-247efa587879 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.551384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.552007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.552283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.552551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] PATH_INFO: `/v3/groups` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.553176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.554649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.569979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 121/234] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Oct 11 06:29:09 2022] POST /identity/v3/groups => generated 229 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:09.569979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.582076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.596107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.605954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04b972-2496-417c-ab35-d62650c2fb1c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.645340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.646046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.646324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.646597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.647280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:09.647640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.649044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.658821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 114/235] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:09.664850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.703241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc0836-a2cf-49fb-9d14-4c0158880192 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.738959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.739635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.739914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.740187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] PATH_INFO: `/v3/users/a32535abe0c2483586b8763c76ee1658` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.756409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] RBAC: Authorizing `identity:get_user(user_id=a32535abe0c2483586b8763c76ee1658)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.757923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.759956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 122/236] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/users/a32535abe0c2483586b8763c76ee1658 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:09.765347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.794205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb16d42-4b33-4f61-8e15-517ea896bf05 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.831746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.832032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.832596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.832875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.833221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] PATH_INFO: `/v3/users/8a29ee29b8f74f549477dc7208b24daf` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.834532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] RBAC: Authorizing `identity:get_user(user_id=8a29ee29b8f74f549477dc7208b24daf)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.836243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.838344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 115/237] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/users/8a29ee29b8f74f549477dc7208b24daf => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:09.843801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:09.860130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:09.870804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db4060-9bf7-4b17-96de-df69e68bff8e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.910723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.911249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.911563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.911838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.912566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] RBAC: Adding query filter params (role.id=81fbc573f2db4065af353a8879e07c88, user.id=a32535abe0c2483586b8763c76ee1658, scope.system=all) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:09.912870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.914312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9e9fd-b7cb-4f0c-8d79-65ec9cbab89e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:09.919496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 123/238] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Oct 11 06:29:09 2022] GET /identity/v3/role_assignments?user.id=a32535abe0c2483586b8763c76ee1658&scope.system=all&role.id=81fbc573f2db4065af353a8879e07c88 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:09.951360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acfe30-8110-4e9d-9ec8-bce74f28239b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:09.989410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9cfcb-38d7-4788-b3b7-413a2aa560ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:09.990104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9cfcb-38d7-4788-b3b7-413a2aa560ef None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:09.990382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9cfcb-38d7-4788-b3b7-413a2aa560ef None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:09.990652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9cfcb-38d7-4788-b3b7-413a2aa560ef None admin] PATH_INFO: `/v3/system/users/8a29ee29b8f74f549477dc7208b24daf/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:09.992156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9cfcb-38d7-4788-b3b7-413a2aa560ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a29ee29b8f74f549477dc7208b24daf, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:09.993564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9cfcb-38d7-4788-b3b7-413a2aa560ef None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:10.004114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 116/239] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Oct 11 06:29:09 2022] PUT /identity/v3/system/users/8a29ee29b8f74f549477dc7208b24daf/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:29:11.749655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613df6b3-a33b-4277-a7d2-6726fd5739ce None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:11.750135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613df6b3-a33b-4277-a7d2-6726fd5739ce None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:11.750135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613df6b3-a33b-4277-a7d2-6726fd5739ce None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:11.751497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 124/240] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:11.756993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8c9ca-53e7-4054-82fb-07d0d381fa51 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:11.757382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8c9ca-53e7-4054-82fb-07d0d381fa51 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:11.757538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8c9ca-53e7-4054-82fb-07d0d381fa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:11.775464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e4d8c9ca-53e7-4054-82fb-07d0d381fa51 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:11.776638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e4d8c9ca-53e7-4054-82fb-07d0d381fa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:11.842661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 117/241] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:12.063016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617677e-5f89-4c70-8a00-427e87cbc95d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.063343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617677e-5f89-4c70-8a00-427e87cbc95d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.063541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617677e-5f89-4c70-8a00-427e87cbc95d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.064998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 125/242] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:12.070399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db546-bde2-4f45-b850-5248ec8670f1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.070682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db546-bde2-4f45-b850-5248ec8670f1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.070951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db546-bde2-4f45-b850-5248ec8670f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.083803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:12.093022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ef8db546-bde2-4f45-b850-5248ec8670f1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:12.094238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ef8db546-bde2-4f45-b850-5248ec8670f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:12.108741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617677e-5f89-4c70-8a00-427e87cbc95d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.168163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 118/243] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:12.211486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0d2f1-5377-4251-bb19-9ebd69de27ee None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.211780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0d2f1-5377-4251-bb19-9ebd69de27ee None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.212053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0d2f1-5377-4251-bb19-9ebd69de27ee None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.213574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 119/244] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:12.217581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617677e-5f89-4c70-8a00-427e87cbc95d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.218106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.218385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.218653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.237897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:12.239230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:12.244698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:12.252659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.253310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.253587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.253853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.254945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:12.256425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:12.259563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 126/245] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:12.263589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.311115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 120/246] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:12.373860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005bac0-5422-43e8-bb90-8a66ec40f235 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.412278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.412886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.413161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.413427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] PATH_INFO: `/v3/groups` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.414060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:12.415693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:12.432210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 127/247] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Oct 11 06:29:12 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:29:12.435937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.508190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:12.547253 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cd38a-d454-4cd6-a280-b61dbe92f776 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.584567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.585458 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.585740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.586010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.586231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.598935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:12.604671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:12.605510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:12.606311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 121/248] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:12.611686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.652471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:12.709361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae009b-a398-4b34-8e40-e4a1b710e60e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.730356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187402c2-e937-40bd-b94c-7e09a98662d7 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.753496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.753496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.754059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.754059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.761229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:12.767953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:12.774087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:12.775162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:12.776173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 128/249] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:12.781042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.781199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.781841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.782180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.782777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.787491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:12.787491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:12.787491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:12.798933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 122/250] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:12.806072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.908597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fcaf5-dc7b-46ff-b402-e09ec5a48a6e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.923280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6191d-70c7-4b91-9e9a-5aeae5252b89 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:12.927438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:12.947910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640135ee-73df-4813-8a39-171883e55852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.948613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640135ee-73df-4813-8a39-171883e55852 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.948970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640135ee-73df-4813-8a39-171883e55852 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.949305 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640135ee-73df-4813-8a39-171883e55852 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.950044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640135ee-73df-4813-8a39-171883e55852 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:12.950335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640135ee-73df-4813-8a39-171883e55852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:12.951744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640135ee-73df-4813-8a39-171883e55852 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:12.966953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 129/251] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:12.970801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:12.972070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:12.972525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:12.972713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640135ee-73df-4813-8a39-171883e55852 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:12.973383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] PATH_INFO: `/v3/users/a32535abe0c2483586b8763c76ee1658` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:12.974717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] RBAC: Authorizing `identity:get_user(user_id=a32535abe0c2483586b8763c76ee1658)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:12.977482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:12.979469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 123/252] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/users/a32535abe0c2483586b8763c76ee1658 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:12.985077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:13.049546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:13.096317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640135ee-73df-4813-8a39-171883e55852 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:13.102369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef5c9a-acb8-436b-8983-d177554691aa None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:13.134146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:13.134797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:13.135219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:13.135525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] PATH_INFO: `/v3/users/8a29ee29b8f74f549477dc7208b24daf` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:13.136874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] RBAC: Authorizing `identity:get_user(user_id=8a29ee29b8f74f549477dc7208b24daf)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:13.138417 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:13.140284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 130/253] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:12 2022] GET /identity/v3/users/8a29ee29b8f74f549477dc7208b24daf => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:13.140793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72903d-08e4-468b-aa67-ff1a147d7361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:13.141502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72903d-08e4-468b-aa67-ff1a147d7361 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:13.141779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72903d-08e4-468b-aa67-ff1a147d7361 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:13.142049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72903d-08e4-468b-aa67-ff1a147d7361 None admin] PATH_INFO: `/v3/system/users/a32535abe0c2483586b8763c76ee1658/roles/81fbc573f2db4065af353a8879e07c88` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:13.145639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72903d-08e4-468b-aa67-ff1a147d7361 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a32535abe0c2483586b8763c76ee1658, role_id=81fbc573f2db4065af353a8879e07c88)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:13.145639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72903d-08e4-468b-aa67-ff1a147d7361 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:13.146929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:13.154977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 124/254] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Oct 11 06:29:12 2022] PUT /identity/v3/system/users/a32535abe0c2483586b8763c76ee1658/roles/81fbc573f2db4065af353a8879e07c88 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:29:13.210896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:13.280775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ef3c7-9d55-4443-be24-8e15df82d704 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:13.328329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:13.329398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:13.329880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:13.330352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:13.331908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] RBAC: Adding query filter params (role.id=2985373b41e749088f96f5c6c9b092c8, user.id=8a29ee29b8f74f549477dc7208b24daf, scope.system=all) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:13.332444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:13.334163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b33bd2-4275-4e95-9903-06f28bcdcf26 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:13.350073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 131/255] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Oct 11 06:29:13 2022] GET /identity/v3/role_assignments?user.id=8a29ee29b8f74f549477dc7208b24daf&scope.system=all&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:14.954063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f4de-6901-4580-9519-d200bf5e7533 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:14.954601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f4de-6901-4580-9519-d200bf5e7533 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:14.954601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096f4de-6901-4580-9519-d200bf5e7533 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:14.956542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 125/256] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:14.962112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559121-6b15-4dcb-ad78-e9eb4ddca984 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:14.962527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559121-6b15-4dcb-ad78-e9eb4ddca984 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:14.962723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559121-6b15-4dcb-ad78-e9eb4ddca984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:14.989076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e1559121-6b15-4dcb-ad78-e9eb4ddca984 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:14.990289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e1559121-6b15-4dcb-ad78-e9eb4ddca984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:15.063722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 132/257] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 06:29:15.246285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c719f-4725-4243-92cc-43d19419495b None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.246466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c719f-4725-4243-92cc-43d19419495b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.246759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c719f-4725-4243-92cc-43d19419495b None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.248320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 126/258] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:15.253965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.254252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.254520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.275373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:15.276826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:15.285145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 133/259] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 06:29:15.570511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c719f-4725-4243-92cc-43d19419495b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.589008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c719f-4725-4243-92cc-43d19419495b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.593854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.594494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.594773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.595040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.596787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.611807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd73e-bddf-42ce-8ff6-63d34c04bd63 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.622957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e1dff-fb24-4116-b7fd-4e87f8586b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.623641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:15.624756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e1dff-fb24-4116-b7fd-4e87f8586b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.625038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e1dff-fb24-4116-b7fd-4e87f8586b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.625304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e1dff-fb24-4116-b7fd-4e87f8586b21 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:15.632616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:15.633498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:15.634251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 127/260] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:15.642584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.653100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e1dff-fb24-4116-b7fd-4e87f8586b21 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-8d1e1dff-fb24-4116-b7fd-4e87f8586b21 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:15.660773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:15.661695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:15.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:15.663031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcdf90-cd9a-4969-833e-095d58cd247e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.664710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 134/261] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:15.666126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322d54-c1f7-47bf-931d-a911c7283b6a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.666580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322d54-c1f7-47bf-931d-a911c7283b6a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.667505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322d54-c1f7-47bf-931d-a911c7283b6a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.669308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 135/262] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:15.670352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c322d54-c1f7-47bf-931d-a911c7283b6a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.672960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.673566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.673842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.674107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.675311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:15.675639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.678399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:29:15.678399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:29:15.678399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:29:15.678399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15836c4-64c7-4780-ad04-7e1b6e6d2bb8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.687679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c322d54-c1f7-47bf-931d-a911c7283b6a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.694284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.695081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.695620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.696088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.697025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:15.697500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.699518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:29:15.699518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:29:15.699518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:29:15.701147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.702748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 128/263] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:15.707096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.707096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.707096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.716988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 136/264] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:15.718265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.730728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284c06a-b2b2-4105-bc83-68bd97a9a7ce admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.737937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.737937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.737937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.737937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.740390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:15.741584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:15.750962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.753533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:29:15.753533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:29:15.753533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:29:15.755144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.757119 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 137/265] 10.222.0.39 () {56 vars in 1201 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:15.758644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.774740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb5c8f-533d-4cad-a33f-eb8b0cec1a94 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.780557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.781466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.781944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.782722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.784720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.787005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.789323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 138/266] 10.222.0.39 () {56 vars in 1201 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:15.790608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.802213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffabac8-57a5-43ef-b902-f0bec00938a3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.815004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.817248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.818024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.818739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.822481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.828795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.833227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 139/267] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:15.834323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.857945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a30bde-4e95-4a2f-93a5-c6f6773e0b79 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.870657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 129/268] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:15.871823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.872700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.873341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.873616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.873887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.880115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.882203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.883898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 140/269] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Oct 11 06:29:15 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:15.887503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138327a-8e12-47be-92b3-4146cb1c4d09 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.889589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:15.894107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.895135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.895640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.896140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394/groups/d7c655d3c62d437da1195979de4b96b1/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.898649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110958db305c4f80ae34050770c56394, group_id=d7c655d3c62d437da1195979de4b96b1, role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.900085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:15.901178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:29:15.901178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:29:15.901178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:29:15.904233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.911180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d3ed6-0d2e-40e9-a9ed-aec129ec8641 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:15.922577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 130/270] 10.222.0.39 () {58 vars in 1463 bytes} [Tue Oct 11 06:29:15 2022] PUT /identity/v3/projects/110958db305c4f80ae34050770c56394/groups/d7c655d3c62d437da1195979de4b96b1/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:15.926962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:15.927648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:15.927929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:15.928203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394/groups/d7c655d3c62d437da1195979de4b96b1/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:15.936397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] RBAC: Authorizing `identity:create_grant(project_id=110958db305c4f80ae34050770c56394, group_id=d7c655d3c62d437da1195979de4b96b1, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:15.937465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:29:15.937465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:29:15.937465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:29:15.939446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122f44a-04f9-42a1-b85e-7188fbf0895a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:15.966017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 141/271] 10.222.0.39 () {58 vars in 1463 bytes} [Tue Oct 11 06:29:15 2022] PUT /identity/v3/projects/110958db305c4f80ae34050770c56394/groups/d7c655d3c62d437da1195979de4b96b1/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:29:16.325681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:16.348273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:16.376268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a101-09ff-43b7-908c-6222328b902b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:16.376578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a101-09ff-43b7-908c-6222328b902b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:16.376815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a101-09ff-43b7-908c-6222328b902b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:16.378454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 142/272] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:16.384230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:16.384751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:16.385382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:16.419685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:16.422176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:16.472992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8af8fb-71e4-477b-8689-b62b5d213274 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:16.527194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 143/273] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 06:29:16.536478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:16.537127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:16.537405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:16.537675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:16.552049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:16.557118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:16.557973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 06:29:16.558746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:16.561077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 131/274] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:16.564524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:16.680962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce57d09-5089-4761-918c-64613afc5e80 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:16.719871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:16.720479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:16.720750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:16.721017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:16.721721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:16.722016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:16.723384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:16.728166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:16.734845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 144/275] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:16.742936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:16.849495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:16.874245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781111a-e69c-44dc-b10d-923320626510 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:16.918111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:16.918745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:16.919068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:16.919345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] PATH_INFO: `/v3/users/a32535abe0c2483586b8763c76ee1658` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:16.920781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] RBAC: Authorizing `identity:get_user(user_id=a32535abe0c2483586b8763c76ee1658)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:16.922279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:16.925197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 132/276] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:29:16 2022] GET /identity/v3/users/a32535abe0c2483586b8763c76ee1658 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:16.930157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:16.948632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:16.961595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10331f9-1c16-488a-b4e0-cc7e09e83136 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:16.966244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:16.966920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:16.967243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:16.967545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:16.980312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:16.986102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:16.986971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:16.987810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:16.989518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 133/277] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:17.008471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:17.024740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc86c9-c3bd-488f-9bee-f831df90a59c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:17.032727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:17.033346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.033678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.033952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.034748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:17.035060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:17.036979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:17.050517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 134/278] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:17.059073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:17.072179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1f63-ad94-475c-8b66-beb103287663 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:17.080737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:17.082100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.082389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.082667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.084865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:17.085940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:29:17.085940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:29:17.085940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:29:17.087278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:17.092311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 135/279] 10.222.0.39 () {56 vars in 1201 bytes} [Tue Oct 11 06:29:17 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:17.103110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:17.113787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836fb4f-91ed-4361-87b3-118c153a13a5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:17.117261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1691447-1890-444b-8cc2-6415809696d7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:17.124541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:17.125254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.125609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.125888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.128651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:17.130840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:17.132847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 136/280] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Oct 11 06:29:17 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:17.140054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:17.154065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03146f-bb2f-4514-8800-b9b2e94ab873 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:17.161388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d05ac-ce4b-4975-9cfd-e6a768f7c369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:17.162048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d05ac-ce4b-4975-9cfd-e6a768f7c369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.162317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d05ac-ce4b-4975-9cfd-e6a768f7c369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.162636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d05ac-ce4b-4975-9cfd-e6a768f7c369 admin admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/groups/d7c655d3c62d437da1195979de4b96b1/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.165575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d05ac-ce4b-4975-9cfd-e6a768f7c369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4712eac5a11d44b7aa936726f9d6d34c, group_id=d7c655d3c62d437da1195979de4b96b1, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:17.168476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d05ac-ce4b-4975-9cfd-e6a768f7c369 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:17.184143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:17.184764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.185043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.185317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.186116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] RBAC: Adding query filter params (role.id=81fbc573f2db4065af353a8879e07c88, user.id=a32535abe0c2483586b8763c76ee1658, scope.system=all) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:17.186423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:17.187927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc2cc-cd35-45bd-be75-7b26cf1b179b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:17.196754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 137/281] 10.222.0.39 () {58 vars in 1463 bytes} [Tue Oct 11 06:29:17 2022] PUT /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/groups/d7c655d3c62d437da1195979de4b96b1/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:29:17.206865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 145/282] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Oct 11 06:29:16 2022] GET /identity/v3/role_assignments?user.id=a32535abe0c2483586b8763c76ee1658&scope.system=all&role.id=81fbc573f2db4065af353a8879e07c88 => generated 510 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:17.904394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdd1f-73fc-4f94-979b-10de89f8d28c None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.905086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdd1f-73fc-4f94-979b-10de89f8d28c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.905086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdd1f-73fc-4f94-979b-10de89f8d28c None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.906276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 138/283] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:17.913496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f81a-e86c-4554-974b-478114368e31 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:17.913780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f81a-e86c-4554-974b-478114368e31 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:17.914052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f81a-e86c-4554-974b-478114368e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:17.936886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1421f81a-e86c-4554-974b-478114368e31 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:17.938082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1421f81a-e86c-4554-974b-478114368e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:18.019444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 146/284] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 06:29:18.089557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f511-5350-4f92-a878-e814e6604e9f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.089884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f511-5350-4f92-a878-e814e6604e9f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.090158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f511-5350-4f92-a878-e814e6604e9f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.091611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 139/285] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:18.100279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.100279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.100279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.105369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:18.120506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:18.122440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:18.130991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 147/286] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 06:29:18.427291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6f511-5350-4f92-a878-e814e6604e9f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.445565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6f511-5350-4f92-a878-e814e6604e9f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.451324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.452212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.452726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.453205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.468378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:18.474418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:18.475268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:18.476058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:18.482780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 140/287] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:18.484296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.494063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e197d-dbd4-4fb2-8f6a-b7e911d36258 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.503324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.503953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.504229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.504499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.505303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:18.512039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.514095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.526081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.527503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 148/288] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:18.533374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.545438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e202c70-952e-4197-98de-e629fa4c73ac admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.545706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c534-19eb-47b0-80ff-cb520e08c78e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.550676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.551405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.551709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.551977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] PATH_INFO: `/v3/groups/68e4e3a9502b42aeb62618f9af20acb1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.558505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] RBAC: Authorizing `identity:get_group(group_id=68e4e3a9502b42aeb62618f9af20acb1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.560586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.562589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 149/289] 10.222.0.39 () {56 vars in 1201 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/groups/68e4e3a9502b42aeb62618f9af20acb1 => generated 229 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:18.563408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.564418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.564890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.565418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.568096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.582408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95374d8-1cd7-435d-af43-cffc391a9357 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.587265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.587929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.588241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.588526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] PATH_INFO: `/v3/projects/4b69df7fd16f47b18a624f52e5487b78` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.590046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] RBAC: Authorizing `identity:get_project(project_id=4b69df7fd16f47b18a624f52e5487b78)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.592252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.594324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 150/290] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/projects/4b69df7fd16f47b18a624f52e5487b78 => generated 344 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:18.595273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:18.600411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:18.602181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:18.603025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:18.607024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:18.608884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 141/291] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 83 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:18.610861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8fc4f-6298-4a97-aeab-f71fec2c49dd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.617416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.618146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.618429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.618706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] PATH_INFO: `/v3/projects/4b69df7fd16f47b18a624f52e5487b78/groups/68e4e3a9502b42aeb62618f9af20acb1/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.620408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.621919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b69df7fd16f47b18a624f52e5487b78, group_id=68e4e3a9502b42aeb62618f9af20acb1, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.625016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.644997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1269c-c90e-400d-bf53-7b662f6226da admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.648769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 151/292] 10.222.0.39 () {58 vars in 1463 bytes} [Tue Oct 11 06:29:18 2022] PUT /identity/v3/projects/4b69df7fd16f47b18a624f52e5487b78/groups/68e4e3a9502b42aeb62618f9af20acb1/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:18.733339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.733897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.734209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.734492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.735204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:18.735603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.737588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.750646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 142/293] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:18.756863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.768278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3057e-fc6c-47f1-8def-86741870d8dd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.773447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.774208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.774491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.774760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.775933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.777958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.779756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 152/294] 10.222.0.39 () {56 vars in 1201 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:18.786612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.797340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84447b-03ab-47d4-acdd-3b01388f7a32 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.802431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cb399-739a-4c07-a0da-b53fc7ecc125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.803137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cb399-739a-4c07-a0da-b53fc7ecc125 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.803447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cb399-739a-4c07-a0da-b53fc7ecc125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.803725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cb399-739a-4c07-a0da-b53fc7ecc125 admin admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.804944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9cb399-739a-4c07-a0da-b53fc7ecc125 admin admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.807386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9cb399-739a-4c07-a0da-b53fc7ecc125 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.809301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 143/295] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Oct 11 06:29:18 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:18.815606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:18.826624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542078b-d4c0-496a-a1ad-2c17a8b68963 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:18.832191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba2212-9036-43e1-82a9-837077089c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:18.832929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba2212-9036-43e1-82a9-837077089c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.833206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba2212-9036-43e1-82a9-837077089c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.833480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba2212-9036-43e1-82a9-837077089c74 admin admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/groups/d7c655d3c62d437da1195979de4b96b1/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.835955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba2212-9036-43e1-82a9-837077089c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4712eac5a11d44b7aa936726f9d6d34c, group_id=d7c655d3c62d437da1195979de4b96b1, role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:18.838803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba2212-9036-43e1-82a9-837077089c74 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:18.859227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa458e0-73c9-4d77-b7db-31f427123a1b None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.859611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa458e0-73c9-4d77-b7db-31f427123a1b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.859853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa458e0-73c9-4d77-b7db-31f427123a1b None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.861698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 144/296] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:18.865861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 153/297] 10.222.0.39 () {58 vars in 1463 bytes} [Tue Oct 11 06:29:18 2022] PUT /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c/groups/d7c655d3c62d437da1195979de4b96b1/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:18.867166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88196b-9fa6-43c4-a8e9-d557564e8e5f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:18.867467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88196b-9fa6-43c4-a8e9-d557564e8e5f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:18.867746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88196b-9fa6-43c4-a8e9-d557564e8e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:18.904724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fc88196b-9fa6-43c4-a8e9-d557564e8e5f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:18.905994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fc88196b-9fa6-43c4-a8e9-d557564e8e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:18.923620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:19.000305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 145/298] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:19.199886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dbea1-377d-4764-bb80-a59de4fa2e16 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:19.200558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dbea1-377d-4764-bb80-a59de4fa2e16 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:19.200558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dbea1-377d-4764-bb80-a59de4fa2e16 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:19.202036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 154/299] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:19.208082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:19.208361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:19.208634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:19.227135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:19.228338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:19.315791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dbea1-377d-4764-bb80-a59de4fa2e16 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:19.318357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 146/300] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:19.457811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dbea1-377d-4764-bb80-a59de4fa2e16 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:19.494518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:19.495219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:19.495519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:19.495792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:19.508469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:19.514007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:19.514838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:19.515779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 155/301] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:29:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:19.523694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:19.647864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:19.675991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdb33c-125e-4b03-b98e-0d0e77216f8c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:19.692873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:19.705435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:19.728368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:19.728996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:19.729272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:19.729542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:19.730215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:19.730515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:19.731916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:19.751086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 147/302] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:19.757484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:19.902215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3179a1e-2450-4e42-8e07-bf8b71a4b227 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:19.958004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16da6bd-6668-459a-adb4-1222aa7654aa None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:19.970288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:19.970981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:19.971262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:19.971592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:19.993732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:19.999290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:20.000269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:20.001048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:20.009356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 156/303] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:20.010334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:20.032775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:20.033391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:20.033671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:20.034451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:20.055776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:20.137653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:20.139136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:20.174724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:20.210149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c244bcc-d8f1-4a9e-a555-019580d11cd0 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:20.263290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 148/304] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:20.270265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:20.310416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:20.311047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:20.311327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:20.311650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:20.312332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:20.312638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:20.314026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:20.314198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:20.328354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 157/305] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:20.335179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:20.381860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cb790-40b7-45ac-b5e6-02a59ea8d318 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:20.420161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:20.420807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:20.421086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:20.421358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:20.422491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:20.425171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:20.427035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 149/306] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:20 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:20.432602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:20.490236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d67301-d838-410b-b04b-7e3fba590f69 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:20.541292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:20.543778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a9da5-0021-4a8d-b40c-12dea8b2735d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:20.545440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46570-e87d-447e-b5cb-4333c4eab1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:20.546301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46570-e87d-447e-b5cb-4333c4eab1cd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:20.546585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46570-e87d-447e-b5cb-4333c4eab1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:20.546866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46570-e87d-447e-b5cb-4333c4eab1cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:20.552325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:20.583528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:20.584165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:20.584446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:20.584719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:20.585945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:20.587703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:20.589653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 150/307] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Oct 11 06:29:20 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:20.595522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:20.653588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a46570-e87d-447e-b5cb-4333c4eab1cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:20.663674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a46570-e87d-447e-b5cb-4333c4eab1cd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:20.703919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa791729-3242-4d1c-8cd0-fa0b73d35a90 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:20.742155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:20.742826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:20.743110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:20.743385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:20.744193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] RBAC: Adding query filter params (group.id=d7c655d3c62d437da1195979de4b96b1, role.id=ade080f5f1cd46b89ad7b9088d1f1654, scope.project.id=110958db305c4f80ae34050770c56394) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:20.744500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:20.745929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:20.752285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 151/308] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Oct 11 06:29:20 2022] GET /identity/v3/role_assignments?group.id=d7c655d3c62d437da1195979de4b96b1&scope.project.id=110958db305c4f80ae34050770c56394&role.id=ade080f5f1cd46b89ad7b9088d1f1654 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:20.847583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 158/309] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 06:29:20.853678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:20.919348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:21.005961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8520d05-b620-44a8-a0bf-e6f687984b2a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:21.036524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6b487-3087-474f-820b-675cffe191c6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.036826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6b487-3087-474f-820b-675cffe191c6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.037076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6b487-3087-474f-820b-675cffe191c6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.042048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 159/310] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:21.048344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:21.048884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.048993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.049164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.049269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.049433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.049528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] PATH_INFO: `/v3/projects/110958db305c4f80ae34050770c56394` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.051289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] RBAC: Authorizing `identity:get_project(project_id=110958db305c4f80ae34050770c56394)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:21.052928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:21.054964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 152/311] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:20 2022] GET /identity/v3/projects/110958db305c4f80ae34050770c56394 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:21.061950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:21.087652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:21.089786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:21.136694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:21.202223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc17bf-8dee-4e7b-a292-be615a2075d4 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:21.216814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 160/312] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:21.217774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:21.250225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:21.251023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.251441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.251813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.252987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:21.254426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:21.256591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 153/313] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Oct 11 06:29:21 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:21.262425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:21.385662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7a4c1-97b4-49fb-a520-b338c748caea None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:21.435388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:21.436303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.436719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.439258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.440427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] RBAC: Adding query filter params (group.id=d7c655d3c62d437da1195979de4b96b1, role.id=2985373b41e749088f96f5c6c9b092c8, scope.project.id=110958db305c4f80ae34050770c56394) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:21.440888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:21.442510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc775bd-f74b-4a2d-bf2d-cf8c3548e516 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:21.455651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 161/314] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Oct 11 06:29:21 2022] GET /identity/v3/role_assignments?group.id=d7c655d3c62d437da1195979de4b96b1&scope.project.id=110958db305c4f80ae34050770c56394&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:21.516050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:21.640512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443023cc-4a1c-4b78-9e4d-9f87b3f732b5 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:21.648210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd88b-3723-4004-8fbd-cdf73294a545 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.648512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd88b-3723-4004-8fbd-cdf73294a545 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.648753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd88b-3723-4004-8fbd-cdf73294a545 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.650446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 162/315] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:21.656515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e5d87-e6a4-4f6a-a318-81f91aa1fe7a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.656794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e5d87-e6a4-4f6a-a318-81f91aa1fe7a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.657049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e5d87-e6a4-4f6a-a318-81f91aa1fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.679582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d29e5d87-e6a4-4f6a-a318-81f91aa1fe7a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:21.680534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d29e5d87-e6a4-4f6a-a318-81f91aa1fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:21.694380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62f6db-2d19-440b-87a4-808ba1400f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:21.695029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62f6db-2d19-440b-87a4-808ba1400f64 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.695310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62f6db-2d19-440b-87a4-808ba1400f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.695782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62f6db-2d19-440b-87a4-808ba1400f64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.703408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:21.709487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62f6db-2d19-440b-87a4-808ba1400f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-ab62f6db-2d19-440b-87a4-808ba1400f64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:21.714483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:21.715567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:21.716414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 154/316] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:21.717251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08b6ab-cd71-4083-b664-d3359e27474f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.717742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08b6ab-cd71-4083-b664-d3359e27474f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.717742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08b6ab-cd71-4083-b664-d3359e27474f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.719148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 155/317] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:21.722465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08b6ab-cd71-4083-b664-d3359e27474f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:21.770546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:21.776928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 163/318] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:21.778426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.778854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.779253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.800127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:21.801549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:21.852262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08b6ab-cd71-4083-b664-d3359e27474f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:21.880306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 164/319] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:21.891931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:21.892537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:21.892808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:21.893077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:21.894273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:21.894562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:21.895944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:21.904192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 156/320] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:29:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:21.911037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:21.997668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.017451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb64a3-d5ed-4ff2-84a0-a84056fc7488 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.048802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.055767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.056377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.056651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.056916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.149915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.151386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:22.173082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.180889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957def2-d9f3-4328-be85-d446c999d9b3 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.216137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.216766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.217053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.217329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.232133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:22.238269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:22.239129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:22.239931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 157/321] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:22.240561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.242401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.261329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 165/322] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:22.262446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.353932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6f98-cdaa-4b5e-b2a3-da6bac0be61b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.377553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291155f-63cf-41c7-9acb-1221671a60e4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.394113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.394743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.395025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.395294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.407076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:22.414296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:22.415239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 158/323] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:22.416434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.422264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.422889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.423168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.423570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.424460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:22.424591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.426335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:22.435650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 166/324] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:22.435989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.436325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.492161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.533448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7aa3-d1b8-43c5-9242-02cca605363d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.560737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72fae-5360-4eff-adad-d75b9afef6cc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.573278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.574132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.574571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.574828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.575983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.577418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:22.579043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 159/325] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:22.579837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.598066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.598685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.598966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.599234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.599942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:22.600540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.601927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:22.611511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 167/326] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:22.613375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.682018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.701174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644270f7-b18b-42b3-bda9-65eeb2f206c8 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.737122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.738286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.738432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68457a-ff91-4c4d-87a4-75772c94a5a4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.738608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.738822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.746488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.781571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.783318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.783318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.783580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.785219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.786674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:22.788315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 168/327] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:22.793756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.827282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.828777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:22.891106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eb078-ae8c-4b53-ae64-1ce7f0ae8dcc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:22.906993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.923864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:22.924027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 160/328] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:22.925356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:22.928086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:22.928366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:22.928638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:22.970070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:22.998295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:22.999786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.022108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8d605-d277-48a5-b2bb-86b68e22de66 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.055777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.056366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.056637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.056904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.057619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] RBAC: Adding query filter params (group.id=d7c655d3c62d437da1195979de4b96b1, role.id=2985373b41e749088f96f5c6c9b092c8, scope.project.id=4712eac5a11d44b7aa936726f9d6d34c) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:23.057922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.059325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.064842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 161/329] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/role_assignments?group.id=d7c655d3c62d437da1195979de4b96b1&scope.project.id=4712eac5a11d44b7aa936726f9d6d34c&role.id=2985373b41e749088f96f5c6c9b092c8 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:23.065740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:23.102419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 169/330] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:23.108239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:23.126583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:23.190767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d873b45-1a92-490d-a86c-98fa61a16475 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.199526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:23.228695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d67af-05a9-435a-95a6-707f88e5fc49 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.229943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.230592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.230885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.231164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] PATH_INFO: `/v3/projects/4712eac5a11d44b7aa936726f9d6d34c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.232319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=4712eac5a11d44b7aa936726f9d6d34c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.233780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.235448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 162/331] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:23 2022] GET /identity/v3/projects/4712eac5a11d44b7aa936726f9d6d34c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:23.241060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:23.272839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.273642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.273925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.274515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] PATH_INFO: `/v3/projects/4b69df7fd16f47b18a624f52e5487b78` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.276090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] RBAC: Authorizing `identity:get_project(project_id=4b69df7fd16f47b18a624f52e5487b78)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.277224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.283483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 170/332] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:29:23 2022] GET /identity/v3/projects/4b69df7fd16f47b18a624f52e5487b78 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:23.285346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:23.354469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c65044-4973-4f2b-860c-5ce986eb6dc6 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.373765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:23.403319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.404020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.404269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.404544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] PATH_INFO: `/v3/groups/d7c655d3c62d437da1195979de4b96b1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.405679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] RBAC: Authorizing `identity:get_group(group_id=d7c655d3c62d437da1195979de4b96b1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.405795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7b8c9-adc4-4980-8dcd-6be161289d58 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.407263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.410017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 163/333] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Oct 11 06:29:23 2022] GET /identity/v3/groups/d7c655d3c62d437da1195979de4b96b1 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:23.414889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:23.455672 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.455672 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.455983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.456082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] PATH_INFO: `/v3/groups/68e4e3a9502b42aeb62618f9af20acb1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.458234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] RBAC: Authorizing `identity:get_group(group_id=68e4e3a9502b42aeb62618f9af20acb1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.458595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.460440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 171/334] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Oct 11 06:29:23 2022] GET /identity/v3/groups/68e4e3a9502b42aeb62618f9af20acb1 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:23.466197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:23.478937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:23.524022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646cf6-117c-40eb-b726-da9e76c618a1 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.556918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.557512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.557788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.558055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.558769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] RBAC: Adding query filter params (group.id=d7c655d3c62d437da1195979de4b96b1, role.id=ade080f5f1cd46b89ad7b9088d1f1654, scope.project.id=4712eac5a11d44b7aa936726f9d6d34c) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:23.559070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.560516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a61cc-8a9f-4440-bee9-9533138bf232 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.564744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fab87-d59b-4957-ba56-572634e66fcc None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:23.566399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 164/335] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Oct 11 06:29:23 2022] GET /identity/v3/role_assignments?group.id=d7c655d3c62d437da1195979de4b96b1&scope.project.id=4712eac5a11d44b7aa936726f9d6d34c&role.id=ade080f5f1cd46b89ad7b9088d1f1654 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:23.600845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:23.601460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:23.601735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:23.602003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:23.602745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] RBAC: Adding query filter params (group.id=68e4e3a9502b42aeb62618f9af20acb1, role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=4b69df7fd16f47b18a624f52e5487b78) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:23.603051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:23.604575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4e9d-e888-459b-9843-2d621bd44d79 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:23.611286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 172/336] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Oct 11 06:29:23 2022] GET /identity/v3/role_assignments?group.id=68e4e3a9502b42aeb62618f9af20acb1&scope.project.id=4b69df7fd16f47b18a624f52e5487b78&role.id=287ae34730df480e9b5436bfab5d48bd => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:26.451714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6beb7f4-9c62-4efc-8dc8-cc379fca538d None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.452249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6beb7f4-9c62-4efc-8dc8-cc379fca538d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.452249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6beb7f4-9c62-4efc-8dc8-cc379fca538d None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.453859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 165/337] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:26.460409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e95dd-b43b-4880-8136-f071461c659f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.460938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e95dd-b43b-4880-8136-f071461c659f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.460938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e95dd-b43b-4880-8136-f071461c659f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.486838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-873e95dd-b43b-4880-8136-f071461c659f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:26.488123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-873e95dd-b43b-4880-8136-f071461c659f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:26.585330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 173/338] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:26.664023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295daf28-9ad5-4334-bd1e-58f13f3d71ed None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.664636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295daf28-9ad5-4334-bd1e-58f13f3d71ed None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.665119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295daf28-9ad5-4334-bd1e-58f13f3d71ed None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.666934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 166/339] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:26.672797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458422-7497-4a5e-a92d-dffefd73bdff None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.673315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458422-7497-4a5e-a92d-dffefd73bdff None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.673810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458422-7497-4a5e-a92d-dffefd73bdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.681345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:26.700848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-75458422-7497-4a5e-a92d-dffefd73bdff None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:26.702320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-75458422-7497-4a5e-a92d-dffefd73bdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:26.774646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192dabe-2f05-4aa5-8e18-f0650bb29160 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.774964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192dabe-2f05-4aa5-8e18-f0650bb29160 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.775199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192dabe-2f05-4aa5-8e18-f0650bb29160 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.780529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 167/340] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:26.781893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716a4b0-bd8b-4416-b69a-9224239ada55 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.782180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716a4b0-bd8b-4416-b69a-9224239ada55 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.782461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716a4b0-bd8b-4416-b69a-9224239ada55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.795306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 174/341] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:26.820457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3716a4b0-bd8b-4416-b69a-9224239ada55 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:26.821681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3716a4b0-bd8b-4416-b69a-9224239ada55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:26.895259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:26.924301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 168/342] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:26.996556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6bed8-489e-4548-88c5-878ce1df1e9e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:26.997153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6bed8-489e-4548-88c5-878ce1df1e9e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:26.997645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6bed8-489e-4548-88c5-878ce1df1e9e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:26.999516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 175/343] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:27.004865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.005146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.005414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.031818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:27.033261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:27.048589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6bed8-489e-4548-88c5-878ce1df1e9e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.131849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 169/344] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:27.133108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.166592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6bed8-489e-4548-88c5-878ce1df1e9e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.205511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.206043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.206318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.206581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.223451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.223451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.223451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:27.242155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:27.243182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:27.244078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:27.244078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.244078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:27.244078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 176/345] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:27.244078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.294040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3e3e-f04f-40e6-892b-c7efc812b0ff None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.306514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:27.344459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.345472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.345974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.346985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.363091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.364631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:27.379575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:27.380453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:27.381295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:27.381295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.381295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:27.385547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6026f1d-5a70-4d9e-8780-900295fd58fd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.385694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 170/346] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:27.386480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.429583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.430705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.431576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.432335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.433181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:27.433710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.436278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.453042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 177/347] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:27.457708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.508522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:27.518480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a3c10-1fdc-457f-8820-2d5e1eed1a69 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.555595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.556542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.556830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.557105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.564580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f1fa-1168-465d-b1d5-5af957cc3335 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.569069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.570494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:27.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:27.588172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:27.588963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:27.588963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.588963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:27.589540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 171/348] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:27.590090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.601856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.602471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.602747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.603022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.603875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:27.603990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.605342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.610446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:27.616691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 178/349] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:27.617492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.710963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea63f0-d83a-4261-9801-07d49612001b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.723360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef0db-49ac-491f-90be-d52629b04a3d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.730098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:27.751935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.752822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.753345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.753753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.754620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.756339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.758623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.759272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.759557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.759830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.770781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.772823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.779971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 172/350] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:27 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:27.780788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:27.794706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:27.795743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:27.796806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:27.796806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:27.796806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:27.796806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 179/351] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:27.798071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.870465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:27.910747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e739c-08ec-4291-9b62-f1e0b484af54 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.911657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a22f-0db5-42b6-8bf1-2611dcb2ade2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:27.946178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.946791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.947066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.947332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.948011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:27.948308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.949687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc63f9-3151-4970-9ef7-f7e97f65ec9f None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.953922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:27.954584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.954814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.955083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.956325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:27.958257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:27.962720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 173/352] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:27.963728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54975e1-921c-41ac-84fb-9d506289ca40 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:27.964005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54975e1-921c-41ac-84fb-9d506289ca40 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:27.964280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54975e1-921c-41ac-84fb-9d506289ca40 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:27.965646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 174/353] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:27.966194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54975e1-921c-41ac-84fb-9d506289ca40 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:27.983782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 180/354] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:27 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:27.984487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:28.003694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:28.109059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21669719-4f72-48c3-976e-255b6fa4c278 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:28.123271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54975e1-921c-41ac-84fb-9d506289ca40 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:28.147896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa1ddc-04b3-4bd3-b624-c64c288c3e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:28.148604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1ddc-04b3-4bd3-b624-c64c288c3e9e None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:28.148949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1ddc-04b3-4bd3-b624-c64c288c3e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:28.149286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa1ddc-04b3-4bd3-b624-c64c288c3e9e None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:28.150010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa1ddc-04b3-4bd3-b624-c64c288c3e9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:28.151678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa1ddc-04b3-4bd3-b624-c64c288c3e9e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:28.162416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:28.174442 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:28.175057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:28.175381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:28.177531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:28.177531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:28.177531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:28.178904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:28.183681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 181/355] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:27 2022] POST /identity/v3/users => generated 248 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:28.185606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:28.185606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:28.185783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:28.195674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 175/356] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:28.203470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:28.215588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:28.217592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:28.313172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:28.321671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 182/357] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:28.351940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bfc8e-8efa-4480-8e7d-9e19925615e3 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:28.397587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:28.398203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:28.398480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:28.398750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:28.399400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:28.400848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:28.441037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 176/358] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:28 2022] POST /identity/v3/users => generated 249 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:28.625985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:28.730533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf280-e61c-4476-ab54-952850aed390 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:28.768963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:28.769674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:28.769950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:28.770215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:28.770898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:28.772384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:28.781604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.sql.core [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 06:29:28.781604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 11 06:29:28.781604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [parameters: {'id': '74b8544863c24627929205c8dd220573', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 11 06:29:28.781604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81485) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Oct 11 06:29:28.798719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return ref.to_dict() Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:29:28.799690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application yield resource Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise self.value Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application session.commit() Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 06:29:28.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.flush() Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application compat.raise_( Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 06:29:28.801399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:29:28.802255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:29:28.803041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [parameters: {'id': '74b8544863c24627929205c8dd220573', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:28.805346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 11 06:29:28.807344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:28.809287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 183/359] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Oct 11 06:29:28 2022] POST /identity/v3/roles => generated 144 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:28.809418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:28.863738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:28.922303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371bc6-f4a9-497e-a9d1-e0e6694ee602 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:28.960665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ee7cc-7928-472d-954f-3182cc3b3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:28.961657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ee7cc-7928-472d-954f-3182cc3b3710 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:28.962150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ee7cc-7928-472d-954f-3182cc3b3710 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:28.962609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ee7cc-7928-472d-954f-3182cc3b3710 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:28.974483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ee7cc-7928-472d-954f-3182cc3b3710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-3f7ee7cc-7928-472d-954f-3182cc3b3710 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:28.980059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:28.980910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 06:29:28.981673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:28.984201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 177/360] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:28.991486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:29.091037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:29.149141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bc83c-a49f-4ad9-8525-2b05a6798a0e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:29.195941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:29.196458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:29.196732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:29.197002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:29.197667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:29.197961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:29.199320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0a92a-69b7-4316-8231-274032a1ef57 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:29.211616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 184/361] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:30.437694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcb735-312a-40a6-a473-e9fc5fc103d6 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:30.438211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcb735-312a-40a6-a473-e9fc5fc103d6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:30.438211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcb735-312a-40a6-a473-e9fc5fc103d6 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:30.439905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 178/362] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:30.445555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d0003-0878-470c-a154-357356021f81 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:30.446030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d0003-0878-470c-a154-357356021f81 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:30.446101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d0003-0878-470c-a154-357356021f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:30.473544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c15d0003-0878-470c-a154-357356021f81 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:30.474723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c15d0003-0878-470c-a154-357356021f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:30.553603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 185/363] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:30.639680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf50bf6-e77d-49fe-aae1-f6d4acef18ca None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:30.639997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf50bf6-e77d-49fe-aae1-f6d4acef18ca None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:30.640227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf50bf6-e77d-49fe-aae1-f6d4acef18ca None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:30.641567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 179/364] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:30.659596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b3116-b79e-431f-ada6-0b800075a60a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:30.659853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b3116-b79e-431f-ada6-0b800075a60a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:30.660058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b3116-b79e-431f-ada6-0b800075a60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:30.683250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0d7b3116-b79e-431f-ada6-0b800075a60a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:30.684480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0d7b3116-b79e-431f-ada6-0b800075a60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:30.756852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:30.779497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 186/365] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:30.838409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb79f4-fdb5-4553-b5c7-49f35639d64c None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:30.838729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb79f4-fdb5-4553-b5c7-49f35639d64c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:30.838963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb79f4-fdb5-4553-b5c7-49f35639d64c None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:30.841067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 180/366] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:30.846995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:30.847283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:30.847635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:30.867294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:30.868491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:30.928520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb79f4-fdb5-4553-b5c7-49f35639d64c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:30.957966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 187/367] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:31.081644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb79f4-fdb5-4553-b5c7-49f35639d64c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.084103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.128594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.129233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.129508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.129775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.148120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:31.152982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:31.153831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:31.154618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 181/368] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:31.160565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.172398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.301990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72478018-a601-4c49-80bf-37d75244ddb0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.305986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b0fc9-1206-426c-ac24-b4f0fd4e15cc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.328839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.343985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.344733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.345006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.345272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.352570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.353322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.353794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.354161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.355005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:31.355445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:31.356914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:31.357332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:31.363658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:31.364913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:31.366580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:31.375465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 188/370] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:31.375656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 182/370] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:31.375656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.375905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.397855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.478328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deb245-ddcf-4d9b-aaf9-b3733c5db5f0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.489784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a2ec-3c8b-44af-9cf8-fd398c18e688 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.514311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.515276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.515725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.516133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.517031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:31.517488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:31.519160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:31.523190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9573-a6f8-47de-9862-06f6e8a1a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.523944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9573-a6f8-47de-9862-06f6e8a1a681 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.524227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9573-a6f8-47de-9862-06f6e8a1a681 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.524499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9573-a6f8-47de-9862-06f6e8a1a681 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.528380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 183/371] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:31.528380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.539739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb9573-a6f8-47de-9862-06f6e8a1a681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-44cb9573-a6f8-47de-9862-06f6e8a1a681 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:31.545020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:31.545941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:31.546774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 189/372] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:31.548114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7ac3f-4a7f-4b7f-80a1-f8509fefa6fe None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.548114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7ac3f-4a7f-4b7f-80a1-f8509fefa6fe None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.548114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7ac3f-4a7f-4b7f-80a1-f8509fefa6fe None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.549148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 190/373] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:31.550191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7ac3f-4a7f-4b7f-80a1-f8509fefa6fe None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.591992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.629091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.645192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93054f00-bec8-44f4-8d98-e5ed83293734 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.656892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7ac3f-4a7f-4b7f-80a1-f8509fefa6fe None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.682353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.682981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.683263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.683795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.693735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.694372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.694652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.694829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:31.694923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.696279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:31.706549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:31.708049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:31.712142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:31.712960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:31.713852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:31.713852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:31.713852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:31.713852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 184/374] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:31.714053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-eef0ea1c-0bca-4a94-9355-f8fad781fda0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:31.725786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:31.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:31.727554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:31.727554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:31.727554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:31.727554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 191/375] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:31.732503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.732789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.733067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.752308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:31.753517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:31.823727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659f94-6573-40d8-b525-1a510c5ab90c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.826004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 192/376] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:31.826933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.857937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.874811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.875723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.876172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.876711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.877992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:31.878527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:31.880500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:31.886526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:31.896054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 185/377] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:31.897670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:31.949150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fb3ca-c2a7-475f-ba3c-160657f4f63f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:31.995145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:31.995789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:31.996063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:31.996328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:31.996967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:31.997265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:31.998637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.014968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 193/378] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:32.015896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.040958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c815ea-83e3-4e27-8a9c-8189cb23e520 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.085603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.086216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.086493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.086767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.087409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:32.087739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.089105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.102012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.104126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 186/379] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:32.105047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.126491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ffc94-1b0d-465f-8859-9521103b7b25 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.159973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.160595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.160869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.161133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.163225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.171304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.172771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-e6b23f05-0c0a-49c8-b20f-ea1efd74f12c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:32.186577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:32.187459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:32.188294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:32.188294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:32.188294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:32.188294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 194/380] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:32.188809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38dfb-e430-4f56-8b59-5a30ac74ba94 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.189087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38dfb-e430-4f56-8b59-5a30ac74ba94 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.189356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38dfb-e430-4f56-8b59-5a30ac74ba94 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.190621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 195/381] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:32.191302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38dfb-e430-4f56-8b59-5a30ac74ba94 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.209728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519df1-5fdf-46ad-b48c-2bac059d9868 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.242465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.243085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.243362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.243698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.250788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.252355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:32.264540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:32.265576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:32.266823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 187/382] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:32.271436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.293918 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38dfb-e430-4f56-8b59-5a30ac74ba94 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.322657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.327689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.328315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.328590 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.328861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.336631 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.338073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:32.350866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:32.351928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:32.353025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 196/383] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:32.353720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.374483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020283de-b937-4bfe-9b08-1dbd8a317b64 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.393804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.409609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384ae5f-062a-4ff9-9045-a0e50869aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.410392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384ae5f-062a-4ff9-9045-a0e50869aa2f None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.410676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384ae5f-062a-4ff9-9045-a0e50869aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.410948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384ae5f-062a-4ff9-9045-a0e50869aa2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.421706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384ae5f-062a-4ff9-9045-a0e50869aa2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-3384ae5f-062a-4ff9-9045-a0e50869aa2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:32.427062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:32.427981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:32.428742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 188/384] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:32.429409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.429770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.429855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.447259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:32.448493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:32.456476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442e8ef-2ae3-4df4-837f-2f1d6f665b7f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.489068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.489672 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.489945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.490491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.490839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:32.491137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.492541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.505046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 197/385] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:32.505855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.512957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 189/386] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:32.513781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.569460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.617395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb26f6b-6e59-41e4-943b-f59339f8351f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.620479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f219e4-00dc-49cc-8160-59db25effb3b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.628877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.653794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.654325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.654600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.654866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.655625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:32.655928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.657346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.661641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.662441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.662925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.663394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.664314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:32.664787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.666430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.696430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 190/387] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:32.697763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.709994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 198/388] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:32.710873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.803588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f96d1-9f15-4c72-b0bb-3648e8f855d0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.812758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a13fca-906d-47e2-867f-6468a0cc3460 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.823885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.837342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.837941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.838217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.838485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.839150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:32.839469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.840849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.847514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.848139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.848411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 191/389] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:32.848502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.848678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.849208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.856825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.858693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:32.871206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:32.872160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:32.873028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 199/390] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:32.873569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:32.881293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:32.950767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d995a-cbec-4462-87e7-49f33d99cc6c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.970064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c443ef-081a-481e-8fad-ff3c7695f209 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:32.983120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:32.983775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:32.984058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:32.984331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:32.995007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:32.996584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.003998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.004611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.004894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.005227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:33.011094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.012001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:33.012812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:33.012812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.012812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:33.012812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 192/391] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.013031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.018883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.020382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:33.034776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.035848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:33.037011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:33.037011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.037011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:33.037011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 200/392] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.037011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.062530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.100063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.120491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c3c15-7807-4489-8486-97b511bcf281 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.138224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b9b63-87ee-46c6-b99a-8527a5f111dd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.154158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.154782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.155060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.155332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.165906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.167360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.172047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.172664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.172941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.173208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:33.181696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.182757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:33.183740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:33.183740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.183740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:33.183740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 193/393] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.183740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.188147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.189603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:33.204554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.205570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:33.206549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:33.206549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.206549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:33.206549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 201/394] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.206549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.281355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.281528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef623fd6-48a1-481e-a407-bd92b6643a82 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.307602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f168692-a104-4466-9a87-8121909ae2e5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.315152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.315797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.316080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.316354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.317085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:33.317391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.318829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.320441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.336801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 194/395] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:33.337592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.345049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.345657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.345937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.346725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.347408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:33.347745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.349109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.360977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 202/396] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:33.361765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.439340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af693885-59a4-4ecb-a886-accb6480169a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.459507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5ec1-471e-42df-a8c5-a44a7204561d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.473983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.474584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.474858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.475126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.475799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:33.476097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.477452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.488329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 195/397] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:33.489103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.492490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.493229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.493507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.493774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.494408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:33.494701 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.496087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.497558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.508232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 203/398] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:33.509082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.540421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.587119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86b7e0-ee45-499c-b791-03ea31b952ee None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.607376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ac4b-0cb9-4377-8582-384cd0a284d1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.621123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.621724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.622006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.622281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.622922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:33.623228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.624623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.637114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 196/399] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:33.637894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.645716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.646341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.646619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.646891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.657204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.658945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:33.672674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.673715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:33.680233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:33.680233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:33.680233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:33.680233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 204/400] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.680233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.725333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.735995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757afc7-1040-4180-b03f-f06fb239f3f6 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.749234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.770320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.770938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.771213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.771513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.777904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.779327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.779603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277669f6-b856-4662-a276-1bd0112a4946 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:33.790850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.791873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 197/401] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.797539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.813098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.813726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.814008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.814285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.821464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.823025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:33.834789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:33.835905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 205/402] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:33.837208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.897709 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f935d-1d5d-4b57-8ba1-7985cadd2e46 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.930208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.930812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.931097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.931371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.932046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.933464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.933561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f26d0a-4f27-41b8-8528-870149fea917 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:33.941352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.953755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 198/403] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:33 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:33.954553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:33.969329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:33.970079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:33.970433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:33.970769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:33.980195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:33.981676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:33.993590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:33.996763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:33.997868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 206/404] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:34.004011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.083340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671be65d-03b2-4f6e-ab8c-9ed2c2e260b1 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.122346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949db798-abff-4f72-9c1f-8dc4e33b75bf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.123429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.124551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.125353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.125940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.126327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:34.127257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.128251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.142269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 199/405] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:34.143388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.168079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.168413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.168771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.169102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.169856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:34.170218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.171736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.190576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 207/406] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:34.191524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.220096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:34.245820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:34.258445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4313-15c0-4fce-91ac-6df55bda4ffb None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.295435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.296222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.296603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.296980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.297687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:34.297989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.300062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.301815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686031-1fe1-403f-874f-0338a15d0e6c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.317809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 200/407] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:34.318637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.349034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.349663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.349916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.350183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.350940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:34.351206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.352675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.375845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 208/408] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Oct 11 06:29:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:34.375845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.429110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e4c10-51f7-4991-bf11-8cfa0c4ca85a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.465165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.465889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.466233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.466569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.473418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:34.473778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.475326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:34.487917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:34.488895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:34.489885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 201/409] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:34.490426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.493861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89d89-2aca-4932-b960-b0c2a3fffbf2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.504167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:34.529853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.530511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.530792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.531068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/463c2cf049454fa281e99b1a864cdf60/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.546132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=463c2cf049454fa281e99b1a864cdf60, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.547691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.560960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 209/410] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:34 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/463c2cf049454fa281e99b1a864cdf60/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:34.572442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.604074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50668-419d-4a35-b1b6-4bbea7f9961b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.644691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.645426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.645707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.646177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/fbf7156a9d9448a6987fe7999543cedb/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.664810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=fbf7156a9d9448a6987fe7999543cedb, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.666468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.690502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 202/411] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:34 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/fbf7156a9d9448a6987fe7999543cedb/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:34.690502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.753914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b83f7-e4e4-4f48-bb66-bdc7a511ceb8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.794032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:34.804436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.805105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.805640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.805640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.808150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:34.818838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.820653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:34.840447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:34.841468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:34.842269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:34.842269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:34.842269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:34.842269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 210/412] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:34.847317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646124-2eaa-402a-816a-2d63d38f99fb None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:34.849197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.894468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:34.895107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:34.895387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:34.895693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:34.896412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:34.896719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:34.900447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:34.921049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 203/413] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:34.925396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:34.991225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d142b-1258-4c13-9e2a-56639a78fe8a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:35.029132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:35.029744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:35.030024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:35.030300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:35.030984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:35.031238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:35.032920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:35.043503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07912146-7df5-40e1-a111-6c9a30408013 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:35.045943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 211/414] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:35.051891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:35.065914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:35.086592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:35.087825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:35.088321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:35.089608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/3b826760f3d747b888ea373cf5021903/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:35.101902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:35.107631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=3b826760f3d747b888ea373cf5021903, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:35.109331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:35.139570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 204/415] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:34 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/3b826760f3d747b888ea373cf5021903/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:35.171171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9d463-5d4e-4ed5-8001-bdbf39a7c007 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:35.215231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:35.216138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:35.216570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:35.216984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:35.227789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:35.229537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:35.243491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:35.244609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:35.245642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:35.247795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 212/416] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:35.253667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:35.418583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa63b4-5db0-47e9-942b-60ce8e6da726 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:35.462627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:35.463248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:35.463608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:35.463885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:35.464588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:35.464891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:35.466325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f33e-0ce4-47a1-8643-8f1aed120ea5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:35.479288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 205/417] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:35.485264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:35.611292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25d39c-6f3e-4c0b-9c54-abb4f840869a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:35.618754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:35.661680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a22a0b-92be-485e-8a0f-4757a79e13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:35.662267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22a0b-92be-485e-8a0f-4757a79e13e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:35.662545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22a0b-92be-485e-8a0f-4757a79e13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:35.662820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22a0b-92be-485e-8a0f-4757a79e13e8 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/0d9f52ea57a54084ac5aaf40f42dba58/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:35.680253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a22a0b-92be-485e-8a0f-4757a79e13e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=0d9f52ea57a54084ac5aaf40f42dba58, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:35.681776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a22a0b-92be-485e-8a0f-4757a79e13e8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:35.708367 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 213/418] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:35 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/0d9f52ea57a54084ac5aaf40f42dba58/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:36.288364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5615fbf-616e-4ca8-af17-738b9b35970d None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:36.288908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5615fbf-616e-4ca8-af17-738b9b35970d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:36.288908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5615fbf-616e-4ca8-af17-738b9b35970d None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:36.290534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 206/419] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:36.295490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:36.295780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:36.296051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:36.318227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:36.319441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:36.411975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 214/420] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:36.834766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5615fbf-616e-4ca8-af17-738b9b35970d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:36.871355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:36.972106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5615fbf-616e-4ca8-af17-738b9b35970d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:37.017394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:37.018648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.018931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.019205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.032838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:37.038352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:37.039216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:37.040025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 207/421] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:37.045603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:37.145521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:37.163204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63dbd3-513f-4152-a9ee-25d8f93715fd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:37.202762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:37.203387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.203695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.203968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.204719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:37.205027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:37.206443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf07e7-31f6-4df9-874a-5130078a0b58 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:37.215449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 215/422] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:37.220650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:37.316232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ace229-9430-4d3c-b334-320f0ee3c7cf None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.316695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ace229-9430-4d3c-b334-320f0ee3c7cf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.317086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ace229-9430-4d3c-b334-320f0ee3c7cf None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.318839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 216/423] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:37.324317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.324598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.324876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.337703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:37.345373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:37.347633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:37.361683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8165e6-8218-4865-8590-e26968d19a8b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:37.428886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:37.429530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.429802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.430080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.452166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:37.453639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-395d311b-b54b-4553-ac59-3a00c626f03c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:37.479026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:37.480012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:37.480824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:37.480824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:37.480824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:37.483444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 217/424] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:37.483577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 208/425] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:37.491671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:37.612761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:37.649872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470a8f2-eb23-41aa-9505-be2d7cddc1ac None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:37.654974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d81f17-23f8-4eca-9ca7-2588853eae12 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.655260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d81f17-23f8-4eca-9ca7-2588853eae12 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.655557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d81f17-23f8-4eca-9ca7-2588853eae12 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.657082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 209/426] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:37.664675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.664675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.664675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.700653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:37.701852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:37.718518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:37.719130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.719408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.719696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.720341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:37.720645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:37.722052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:37.741642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 218/427] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:37.748524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:37.799340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 210/428] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:37.800578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:37.861991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f862ab-6062-41ce-b964-434dbbfea66f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:37.880673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:37.901494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:37.902464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:37.902940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:37.903578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:37.920234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:37.922183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:37.934495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177f906-6c9d-4ce6-a871-d722f6de2a33 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:37.944457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:37.948402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:37.949313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:37.950183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:37.953668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 219/429] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:37.960460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.007622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.008688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.009181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.009645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.032568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:38.039447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:38.040314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:38.041075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 211/430] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:38.047467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.128822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd5a37-b955-4a2d-be37-254b763378aa None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.177072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.177774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.178098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.178310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.179100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:38.179406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.180855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec999a2-56d6-41a4-aa5d-378a1a095af4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.187512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608884d6-19fb-4f40-8a31-3c870fa4f912 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.199194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 220/431] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Oct 11 06:29:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:38.201420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f83229-64fb-4951-a026-278c00e6af6b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.201906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f83229-64fb-4951-a026-278c00e6af6b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.202537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f83229-64fb-4951-a026-278c00e6af6b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.204646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 221/432] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:38.205636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f83229-64fb-4951-a026-278c00e6af6b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.221573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.227900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.228510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.228787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.229062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.229725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:38.230028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.231386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.240916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 212/433] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:38.241794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.271298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.320205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f83229-64fb-4951-a026-278c00e6af6b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.356709 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c8559-a75a-40b7-8c73-063904cd13b9 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.361314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddf6cc-54bd-40af-9dee-2960f6c9a237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.362538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddf6cc-54bd-40af-9dee-2960f6c9a237 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.362817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddf6cc-54bd-40af-9dee-2960f6c9a237 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.363089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddf6cc-54bd-40af-9dee-2960f6c9a237 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.375489 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddf6cc-54bd-40af-9dee-2960f6c9a237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-5cddf6cc-54bd-40af-9dee-2960f6c9a237 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:38.380913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:38.381764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:38.382537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 222/434] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:38.383278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.383715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.383715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.403484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:38.403484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:38.406316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.407039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.407383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.407962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.420338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.421821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:38.436802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:38.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:38.438592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:38.438592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:38.438592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:38.438592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 213/435] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:38.439329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.479046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 223/436] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:38.479912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.488522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.527858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.545209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d65e4e-08a0-4c8a-bcd7-d022dc1eac83 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.580048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.580797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.581081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.581356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.587798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125b07c-b79b-4d30-9314-a8684c2457fa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.592940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.594435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:38.609048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:38.609897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:38.610712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:38.610712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:38.610712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:38.610712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 214/437] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:38.610930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.627279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.628152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.628605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.629161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.630130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:38.630503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.631899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca92d7-4c57-413b-8cac-f993030d4fb6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.641041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 224/438] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:38.642229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45b8e4-d6bd-416d-b672-01be44712b54 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.642510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45b8e4-d6bd-416d-b672-01be44712b54 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.642780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45b8e4-d6bd-416d-b672-01be44712b54 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.644207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 225/439] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:38.644832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45b8e4-d6bd-416d-b672-01be44712b54 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.711634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.718143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df74b6-43b8-4d74-8fe1-ed456ba19c88 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.751361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45b8e4-d6bd-416d-b672-01be44712b54 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.754117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.755205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.755514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.755788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.756433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:38.756736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.758119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.760582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.770913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 215/440] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:38.771753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.790112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.790713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.790981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.791241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.791909 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:38.792204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.793557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb740-611f-406d-8f2e-e952f8605ce0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.805869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 226/441] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:38.807063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.807343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.807637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.824268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:38.825425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:38.870297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a700954-1192-4ca0-9e65-420ae010b73b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:38.899491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 227/442] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:38.899491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.914921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:38.915590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:38.915894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:38.916137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:38.926507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:38.928082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:38.943166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:38.944062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:38.944900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:38.944900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:38.944900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:38.944900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 216/443] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:38.945117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:38.957202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.991255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:38.995043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd046d-353f-4578-ac60-e25df7115c07 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.030481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.031112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.031389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.031687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.042378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:39.047214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:39.048232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 228/444] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:39.049133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.060835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a906518-453b-428c-b67f-ee47a92bf0f2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.097604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.098286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.098571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.098936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.106252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.107713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:39.119789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:39.120599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:39.121468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 217/445] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:39.121917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.162622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756599d-d6d5-4e09-bd10-f706f817b5a9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.190580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.196770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.197396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.197673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.197946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.204897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.206903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:39.218168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:39.219044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 229/446] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:39.220464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.227392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa676782-6196-469e-bb1b-378010eb002a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.239403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.261085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.261624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.261904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.262175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.262815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:39.263115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.265496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.278124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 218/447] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:39.278911 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.323164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77221a-36bf-49bb-81c7-5e79ba73daca None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.356047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.356643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.356926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.357194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.357846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:39.358144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.359525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 230/448] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:39.367910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.380475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2f628-25d8-4996-960a-9d471ad7b3e7 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.406896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.415560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.416167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.416445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.416717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.417409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:39.417712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.419146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.431531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 219/449] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:39.431531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.453150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.472702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87453c-78f1-48b5-9df9-bc63b6c9ade3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.504880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.505496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.505770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.506035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.515886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:39.521521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:39.522551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:39.523349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 231/450] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:39.524010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.531224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8e470-82e3-4611-832a-ed7a4e2273c3 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.563954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.564558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.565363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.565632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.566355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:39.566658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.568112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.585939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 220/451] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:39.586727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.624831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21357ba7-4dfd-4e74-b14d-91f7a3bee8c9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.630104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.659345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.659991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.660267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.660530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.667618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.669050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.671465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:39.680436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:39.681491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 232/452] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:39.682452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.686226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dd364-1399-497d-8557-3ebeafc2607d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.719359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.720011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.720289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.720560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.730292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.731740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:39.746330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:39.747240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:39.748094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:39.748094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:39.748094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:39.748094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 221/453] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:39.748323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.786323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a149e-a05e-44e8-a1a2-c36b54c7d736 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.819434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.820123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.820401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.820671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/498356fa9eaa41e09da95e0a151105dd/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.834676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=498356fa9eaa41e09da95e0a151105dd, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.836141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.842876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.847063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37acb0-ca2f-4d2f-8621-283a338dc8ca None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:39.848331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 233/454] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:39 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/498356fa9eaa41e09da95e0a151105dd/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:39.849104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.884101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:39.884879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:39.885258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:39.885603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:39.886389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:39.886760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:39.888299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:39.901979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 222/455] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:39.901979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:39.930689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:39.993158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05056eab-2e26-48f2-9327-0362355918cf None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.014083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818506d6-7a4e-4a18-b086-f5c2a8e8328c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.035472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.036491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.036766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.037045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.054895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.055528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.055807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.056075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.056796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:40.057101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.058506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.076746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 223/456] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:40.077555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.100590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.113448 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.115124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.169837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.187276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d897017-514d-4828-8866-ae7f55fc8961 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.222014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.222619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.222893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.223154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.223815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:40.224110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.225471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.232154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 234/457] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:40.233202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.241262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 224/458] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:40.242119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.324906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.349734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aac0e-8651-4722-9ca4-96b24bd5af7c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.350277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31d100-880b-4b81-8907-35e81bf8533e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.383885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.384808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.385089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.385358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.386719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.387459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.387742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.388033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.396254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.397724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:40.419252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:40.420377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:40.421192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:40.421192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:40.421192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:40.421192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 235/459] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:40.421192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.423977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.471783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.473235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.525891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5cc45-bf48-4a5f-80bf-cd376b70b663 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.553599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.564287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.565494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.565775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.566049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.581283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 225/460] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:40.581283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.582437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.584007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:40.599626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:40.600658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:40.601478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:40.601478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:40.601478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:40.601478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 236/461] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:40.601893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.667649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.685038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7313e08-3922-438e-88db-38eb0d87f90c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.716097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13acbbf-346d-43b3-8d38-154ce6119122 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.719565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.720216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.720488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.720762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.729835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.731295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:40.745633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:40.746537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 11 06:29:40.747434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:40.749159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 226/462] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:40.750061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.759247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.760302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.760602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.760872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.761510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:40.761809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.763855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.776326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 237/463] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:40.777081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.796588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.863683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a698e10-dbdf-4ba9-85e9-641c179a9c8e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.878946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4737-3f13-4d9b-a9f9-044c320724eb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:40.898094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:40.901405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.902038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.902306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.902570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.914303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.915787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.923162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:40.924870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:40.925548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:40.925964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:40.926748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:40.927210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:40.928763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:40.930163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:40.931184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:40.932112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:40.932112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:40.932112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:40.932112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 227/464] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:40.932418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:40.945194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 238/465] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:40.946013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.031851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.043592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecd629-ffa8-49f3-b6a0-ba1863b1c7f0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.051632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c9b03-a582-4a4d-ae7b-ea1a39bfaad3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.080062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.081438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.081713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.081984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.082711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:41.083012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.084496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.090083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.090962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.091528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.091974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.101389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.103031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.104112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 228/466] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:41.105005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:41.120438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:41.121519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 06:29:41.122587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:41.123579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 239/467] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:41.124384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.160498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.222244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe7009-4b51-4633-8857-bcca17c152c8 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.230346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145e37-11cc-478b-a03e-110d2b8bf193 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.258904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e7352-de97-4772-abda-1703836a121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.259858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e7352-de97-4772-abda-1703836a121c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.260204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e7352-de97-4772-abda-1703836a121c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.260629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e7352-de97-4772-abda-1703836a121c None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.261447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e7352-de97-4772-abda-1703836a121c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:41.261813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e7352-de97-4772-abda-1703836a121c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.263191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e7352-de97-4772-abda-1703836a121c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.266182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.266867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.267206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.267570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.275056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.275982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 229/468] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:41.276612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e7352-de97-4772-abda-1703836a121c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.281214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.289211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:41.294509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:41.295512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:41.296450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:41.300320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 240/469] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:41.300999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.391245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.391837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e7352-de97-4772-abda-1703836a121c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.399186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9185b-58ae-47e4-8970-6cc89f156b61 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.434414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.435020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.435296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.435597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.438030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.438631 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.438911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.439188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.440487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:41.440791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.442210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.465040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 241/470] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:41.465040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.516580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.518646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.530936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.566514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff954af-c098-4b40-9d36-42aff43db2d9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.601666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.602298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.602579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.602848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.609609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.611062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:41.622638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:41.623693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:41.624530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 242/471] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:41.624970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.636822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 230/472] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:41.636822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.644578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.734642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f958e-f582-4535-9d58-b7b83c839142 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.741211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f734709-c70e-4adb-b430-e15d10a51cff None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.751515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.770143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.771532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.772084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.772533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.773981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:41.774758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.777027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.781117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.781714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.781988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.782256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.792515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 243/473] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:41.793938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.851257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.852598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.880494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:41.913036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7041b-a475-4ad1-8bed-691a15efef2f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:41.952225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:41.952837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:41.953112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:41.953384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:41.954075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:41.954377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:41.956113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:41.966705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 231/474] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:41.966705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.976908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 244/475] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:41.976908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:41.979906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.071443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ca872-5527-498f-aa6a-5255730a1116 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.074578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bfbf-121f-41ed-bef2-c2a426273afd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.103764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.105874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.106521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.106795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.107061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.109537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12896571-4d0e-4038-be73-75f578545252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.110132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12896571-4d0e-4038-be73-75f578545252 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.110406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12896571-4d0e-4038-be73-75f578545252 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.110672 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12896571-4d0e-4038-be73-75f578545252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.111396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12896571-4d0e-4038-be73-75f578545252 None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=463c2cf049454fa281e99b1a864cdf60) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:42.111756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12896571-4d0e-4038-be73-75f578545252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.113172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12896571-4d0e-4038-be73-75f578545252 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.116870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.118270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.118857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 245/476] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/role_assignments?user.id=463c2cf049454fa281e99b1a864cdf60&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:42.119648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12896571-4d0e-4038-be73-75f578545252 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:42.133390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:42.134352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:42.140276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:42.140276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:42.140276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:42.140276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 232/477] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:42.140276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.210387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.244159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12896571-4d0e-4038-be73-75f578545252 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.260422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0099bea-a81c-4a41-99d1-8762b155b764 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.284121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.285217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.285488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.285883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.298136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.300022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.300255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.301312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.301312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.301312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.301916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=fbf7156a9d9448a6987fe7999543cedb) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:42.302218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.303777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03051c6f-2dee-42ac-bf6c-2bf67f257b58 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.309948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 233/478] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/role_assignments?user.id=fbf7156a9d9448a6987fe7999543cedb&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:42.310956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a824f76-c8ce-4203-a108-2590e224a5bd None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.311239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a824f76-c8ce-4203-a108-2590e224a5bd None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.311548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a824f76-c8ce-4203-a108-2590e224a5bd None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.312892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 234/479] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:42.317499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a824f76-c8ce-4203-a108-2590e224a5bd None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c90e82d6-adb3-444e-b9c3-bc7dabd1a10b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:42.318665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:42.319552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:42.320504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:42.320504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:42.320504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:42.320504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 246/480] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:42.326512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.326512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.326512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.347450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:42.348703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:42.432783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.439696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a824f76-c8ce-4203-a108-2590e224a5bd None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.466877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.474561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 247/481] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:42.479566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.490508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.491408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.491714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.492036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.492694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:42.495151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.496685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.514027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 235/482] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:42.522093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.605703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a97f0-587f-4445-a4ec-ee5c36b060fa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.630935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa46879-7783-42cc-9807-039c7c090def None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.644462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.645081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.645353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.645621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.646366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:42.646670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.648193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.663773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 248/483] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:42.670944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.671755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.672130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.672500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.672871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.680039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.681613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:42.695453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:42.696526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:42.697513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:42.699447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 236/484] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:42.706554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.723966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.760095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.802636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb328e-9175-4404-af9b-7560465d6cc1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.819386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68f46-cc75-44de-a632-6c05618f202d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.840159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.841055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.841185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.841406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.849573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.851129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.856670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:42.857599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:42.858085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:42.858551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:42.859462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:42.859964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:42.861628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:42.863474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:42.864337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:42.865155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 249/485] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:42.866008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.877514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 237/486] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:42.878697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:42.976603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:42.981296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba402481-4e5e-48c1-bf7f-c82210a7aa4e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.982307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1927d-6dfd-46d8-9597-69b0e0095fa3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:42.984657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:43.017023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.018034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.018633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.019184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.020153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:43.020596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:43.021522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.022259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.022597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:43.022706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.023048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.035123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:43.036474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 238/487] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:43.037497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:43.041614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:43.042535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:43.043289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 250/488] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:43.048486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:43.147780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceedcd5-89c5-4a03-b3b0-12e2f0b85b1b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:43.175204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66507be1-90fd-4ce9-b963-77c5beed7875 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:43.195043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.195765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.196108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.196449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.197241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=3b826760f3d747b888ea373cf5021903) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:43.198115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:43.200052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:43.208196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 239/489] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:43 2022] GET /identity/v3/role_assignments?user.id=3b826760f3d747b888ea373cf5021903&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:43.209689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:43.225122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.226058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.226541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.227044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.228023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=0d9f52ea57a54084ac5aaf40f42dba58) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:43.228519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:43.230185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:43.242739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 251/490] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:43 2022] GET /identity/v3/role_assignments?user.id=0d9f52ea57a54084ac5aaf40f42dba58&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:43.261281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:43.342872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df290f7-872d-43a6-9d53-fa38bd949e92 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:43.380727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.381657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.382145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.382588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.383505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:43.384008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:43.385604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:43.396104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 240/491] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:43.404584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:43.446602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:43.559710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3347926-7cd0-460d-a4a0-bd092f7bc583 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:43.602227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.602852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.603124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.603387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.615234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:43.616781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:43.633150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:43.634072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:43.635309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:43.635309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:43.635309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:43.635309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 252/492] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:43.641016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:43.742123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:43.754067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f50651-4253-4212-9b28-673be74af7a8 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:43.790191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:43.791347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:43.791869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:43.792426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:43.793256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:43.793750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:43.795382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:43.812811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 241/493] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:43.818087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:43.936470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:43.967960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42afe3-ea23-4066-9b11-c9488bc66e47 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:44.016554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:44.017182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:44.017455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:44.017829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:44.028224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:44.029709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:44.044811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:44.045734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:44.046640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 253/494] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:44.051881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:44.173447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb81f-446b-4131-bda6-cc3fcde28135 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:44.208132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:44.208738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:44.209016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:44.209283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:44.210009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:44.210309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:44.211752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:44.228878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 242/495] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Oct 11 06:29:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:44.233993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:44.406360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ed7e2-b55f-4776-9397-31dade35bec7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:44.452434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347a54f-3a83-4baa-8c81-7c8aa6d1054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:44.452962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347a54f-3a83-4baa-8c81-7c8aa6d1054c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:44.453234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347a54f-3a83-4baa-8c81-7c8aa6d1054c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:44.453499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347a54f-3a83-4baa-8c81-7c8aa6d1054c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:44.457697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:44.532351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347a54f-3a83-4baa-8c81-7c8aa6d1054c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:44.534651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347a54f-3a83-4baa-8c81-7c8aa6d1054c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:44.646361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 254/496] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 06:29:44.652506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:44.666702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:44.768439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b35742-3c88-43d3-8b20-2bc32edeacdd None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:44.803964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:44.804596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:44.804872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:44.805141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:44.815461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:44.816904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:44.823811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b87f-95da-4402-a4ab-8b60029dd365 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:44.824347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b87f-95da-4402-a4ab-8b60029dd365 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:44.824812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1b87f-95da-4402-a4ab-8b60029dd365 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:44.826419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 255/497] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:44.831910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:44.832884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:44.833729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:44.833729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:44.833729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:44.833729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 243/498] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:44.834764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e5ccb-ca73-4072-aa6d-229600eac9cf None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:44.835043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e5ccb-ca73-4072-aa6d-229600eac9cf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:44.835309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e5ccb-ca73-4072-aa6d-229600eac9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:44.838171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:44.872477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3e5e5ccb-ca73-4072-aa6d-229600eac9cf None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:44.873688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3e5e5ccb-ca73-4072-aa6d-229600eac9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:44.919874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:44.955080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:44.983657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e78f-c0a6-4f47-9182-0949a8a8eb69 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.035831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 256/499] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:45.037470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195964c-511f-434f-81a1-d45be42a5089 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.038059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195964c-511f-434f-81a1-d45be42a5089 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.038595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195964c-511f-434f-81a1-d45be42a5089 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.040329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 257/500] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:45.043809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.044654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.045135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.045621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.046457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:45.046991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.048601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.056877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.057451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.057921 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.066822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 244/501] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:45.068158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.100414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:45.102033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:45.213678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640fc64-6c25-4c9a-8cf6-e6cf494d288e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.244524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 258/502] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:45.266134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:45.272611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af69289-beea-4208-9132-c7283decf627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.273310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69289-beea-4208-9132-c7283decf627 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.273595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69289-beea-4208-9132-c7283decf627 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.273898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69289-beea-4208-9132-c7283decf627 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.283141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af69289-beea-4208-9132-c7283decf627 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.285141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af69289-beea-4208-9132-c7283decf627 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-5af69289-beea-4208-9132-c7283decf627 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:45.301715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:45.302976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:45.303869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 245/503] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:45.311034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.314603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af69289-beea-4208-9132-c7283decf627 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.504172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af69289-beea-4208-9132-c7283decf627 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.521774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:45.541679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4b61f-8711-4a69-b5aa-5e5c63d042d5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.551675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.552680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.553158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.553640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.554421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.556053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:45.572851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:45.573850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:45.574739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:45.576758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 246/504] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:45.583464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.601070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.601679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.601950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.602216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.602900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:45.603198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.604661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.617428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 259/505] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:45.618311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.635848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:45.715355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53da1-0437-4d4b-9fd0-4ba2a8b661de None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.722366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8bf3-497d-4091-aaa9-b50b7af408e7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.758817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.759034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.759542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.759824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.759952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.760096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.760368 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.760729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.761449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:45.761838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.763291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.769822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 260/506] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:45.770615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.773537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:45.778836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:45.779990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:45.780787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:45.783653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 247/507] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:45.784426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.796387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:45.894224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cd1e7-7b03-4621-89fa-2e821f75b204 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.897604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:45.899511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dae9e-30d2-465c-ac8b-925141bd5f1d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:45.938136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.938944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.939439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.940275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.941030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:45.941655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.942951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.947779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:45.949000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:45.949449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:45.949870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:45.959463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 248/508] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:45.959463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:45.959764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=498356fa9eaa41e09da95e0a151105dd) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:45.959764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:45.959764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5472cb-f59f-4690-968c-0638caf3cfd1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:45.974260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 261/509] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/role_assignments?user.id=498356fa9eaa41e09da95e0a151105dd&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:46.073282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b34b9-50db-4760-a538-2966531ab9d5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.073661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b34b9-50db-4760-a538-2966531ab9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.073829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b34b9-50db-4760-a538-2966531ab9d5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.079677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 262/510] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:46.081048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f019-7221-42c4-a504-8423981b6b45 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.081330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f019-7221-42c4-a504-8423981b6b45 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.081605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f019-7221-42c4-a504-8423981b6b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.101970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6382f019-7221-42c4-a504-8423981b6b45 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:46.103185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6382f019-7221-42c4-a504-8423981b6b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:46.122370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774045c-1228-4d88-9243-1fdbd0c9c50a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:46.170884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:46.171454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.171740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.172011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.172765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:46.173067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:46.175116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc0141-f153-4788-be0b-def3f1ccdacb None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:46.191785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 249/511] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:46.191980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3742487-1577-4c00-8601-0ea9c2a93ccc None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.192223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3742487-1577-4c00-8601-0ea9c2a93ccc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.192495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3742487-1577-4c00-8601-0ea9c2a93ccc None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.194906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 250/512] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:46.194906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3742487-1577-4c00-8601-0ea9c2a93ccc None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.195778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 263/513] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 06:29:46.199390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.199704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.199977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.221976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:46.223969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:46.232747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:46.251727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:46.310147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 264/514] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:46.338759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3742487-1577-4c00-8601-0ea9c2a93ccc None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:46.374374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:46.375241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.375738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.376153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.387869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:46.389580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:46.406608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:46.407467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:46.409187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:46.409187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:46.409187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:46.409498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 251/515] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:46.417163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.491190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.583614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae22931-94b6-42b2-a206-ee7c5d8686d9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:46.595928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:46.615757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc1530-d997-4827-909f-e91a79b7b8f6 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:46.654255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:46.654890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.655168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.655463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] PATH_INFO: `/v3/services/network` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.656045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:46.657403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:46.659815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:46.660650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.661142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.661609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.662447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:46.662945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:46.664600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:46.666811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:46.667759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:46.669132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 252/516] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:46.669980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.680908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 265/517] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:46.682174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.707923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:46.777899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85420528-cacb-4839-9856-5147a2c71746 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:46.814129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bafcf-5f44-4e2d-9a95-19a6e1e9dd7e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:46.829447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:46.830675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.831600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.832320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.848079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:46.851219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:46.861167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:46.864392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:46.865392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:46.865728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:46.866094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:46.866848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] RBAC: Adding query filter params (name=network) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:46.867215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:46.868686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:46.873546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:46.874457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:46.875325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:46.875325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:46.875325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:46.875526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 266/518] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:46.875526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.877651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 253/519] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:46.879746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:46.957342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:47.001852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fef14-4540-477f-bd4c-462687eb388e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.011708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3e94-bfad-459b-8520-f2c3c9389349 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.043970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.044937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.045680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.046157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.047728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] RBAC: Adding query filter params (type=network) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:47.048234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.050590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.055547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.056038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 254/520] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:47.056145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.056422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.056741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.057666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:47.066960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.068482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:47.085467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:47.086470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:47.087381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 267/521] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Oct 11 06:29:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:47.094661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:47.132329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:47.177197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc3295-cba3-40a4-b537-923cdbd90e58 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.225059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.226981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.227710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.229800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.230739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:47.232492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.235883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.259886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 255/522] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:47.265599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:47.272580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe6537-da3c-4f7c-a44b-7010fb9eed56 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.310104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:47.321822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.323253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.323767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.324238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.325535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:47.326347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.329217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.353617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 268/523] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Oct 11 06:29:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:47.359269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:47.387540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0903103-55f4-48e9-8eb7-0cf34166a5f8 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.413013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:47.426776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.427390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.427710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.428025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.428667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.430095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.455733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 256/524] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:29:47 2022] POST /identity/v3/users => generated 254 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:47.473088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1023b7-ea32-4b68-b8ff-334fe9b59207 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.511919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.512563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.512843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.513116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.525588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.527053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:47.542829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:47.543865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:47.544723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:47.544723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:47.544723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:47.544723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 269/525] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:47.550319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:47.702758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201ec7a-2934-4fda-b232-4bdeca0a1e85 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.736452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.737062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.737339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.737603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.738242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:47.738539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.739931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.752759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 257/526] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:47.758149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:47.802293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:47.870010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753dc61-9d54-4705-99bc-8498fdc5c7fa None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:47.916762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:47.917393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:47.917668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:47.917935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:47.928654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:47.930092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:47.945500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:47.946394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:47.947267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 270/527] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:47.952219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:48.003728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:48.081746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576f1b4-3535-49eb-9fd5-a9ddc8209aaa None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:48.126376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:48.127017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:48.127295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:48.127592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:48.128284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:48.128577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:48.130016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d54c4-3c48-43f1-8698-f1d24b8af750 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:48.146821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 258/528] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:48.152255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:48.274056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:48.281076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6e9dc-661d-4fbf-96f3-9f9ac1b098d8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:48.329425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981eb88-6d90-4121-9a94-f5c79283286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:48.330110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981eb88-6d90-4121-9a94-f5c79283286c None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:48.330389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981eb88-6d90-4121-9a94-f5c79283286c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:48.330657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981eb88-6d90-4121-9a94-f5c79283286c None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/fbf7156a9d9448a6987fe7999543cedb/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:48.332620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981eb88-6d90-4121-9a94-f5c79283286c None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=fbf7156a9d9448a6987fe7999543cedb, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:48.334049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981eb88-6d90-4121-9a94-f5c79283286c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:48.356067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 271/529] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:48 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/fbf7156a9d9448a6987fe7999543cedb/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:48.781934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e035-cdde-467f-b017-b4a1d75cc6e4 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:48.783006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e035-cdde-467f-b017-b4a1d75cc6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:48.784074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e035-cdde-467f-b017-b4a1d75cc6e4 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:48.785607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 259/530] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:48.791440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:48.792265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:48.792735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:48.811994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:48.813490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:48.836750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6cd0-2b4b-4056-8eb8-9b26e9bde572 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:48.837065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6cd0-2b4b-4056-8eb8-9b26e9bde572 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:48.837291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6cd0-2b4b-4056-8eb8-9b26e9bde572 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:48.838893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 260/531] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:48.844122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:48.844412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:48.844694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:48.866711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:48.867951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:48.916369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 272/532] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:48.973073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 261/533] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:49.281142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:49.321086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:49.363775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:49.395912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:49.419100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871920c3-db00-41a2-8bc9-27bb3e26c73f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:49.488496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:49.489134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:49.489410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:49.489682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:49.493827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b436026-802d-4eac-b781-1d34c3dc48b7 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:49.512043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:49.518141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:49.519205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:49.520059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 273/534] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:49.525490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:49.566628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:49.567166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:49.567465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:49.567743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:49.568400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:49.569775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:49.599002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 262/535] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:29:49 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:29:49.703990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c321587-2c5a-4165-a0ba-7d9a1ac8593c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:49.769433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:49.770144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:49.771153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:49.771442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:49.772139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:49.772430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:49.773839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f14f2-91fc-4aa2-a8dc-412cf45f1b87 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:49.788904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 274/536] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:49.794615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:49.893074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:49.906484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc13705-b52a-42f1-b9c5-9fd84da8411a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:49.926721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325049c-4d76-474f-87d0-119bee1538ca None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:49.927721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325049c-4d76-474f-87d0-119bee1538ca None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:49.928282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325049c-4d76-474f-87d0-119bee1538ca None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:49.929882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 275/537] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:49.935860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1689a-b173-415e-b2db-fb22817c66f6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:49.936196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1689a-b173-415e-b2db-fb22817c66f6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:49.937331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1689a-b173-415e-b2db-fb22817c66f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:49.945816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:49.948559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:49.949213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:49.949499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:49.949776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:49.958691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c6f1689a-b173-415e-b2db-fb22817c66f6 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:49.960698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c6f1689a-b173-415e-b2db-fb22817c66f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:49.961741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:49.963190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:49.979806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:49.981155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:49.981997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:49.981997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:49.981997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:49.985874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 263/538] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:49.987671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.050657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 276/539] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:50.052109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d22843-001f-4499-9e77-298d4fdd7f69 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.052394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d22843-001f-4499-9e77-298d4fdd7f69 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.052724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d22843-001f-4499-9e77-298d4fdd7f69 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.054215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 277/540] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:50.059712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.059993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.060264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.101221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:50.105046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:50.139880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ffcea-c700-4201-85d8-d42811045dfe None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.208696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:50.209281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.209563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.209835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.210487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:50.210789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:50.212176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:50.233454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:50.240590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 264/541] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:50.242116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 278/542] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 06:29:50.253163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.346891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.389314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff9471-0f13-4133-bc08-2cc5f8ba760a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.444997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:50.460814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:50.461386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.461660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.461926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.480460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:50.481905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:50.488213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43928a-aaf9-4e20-b5d3-798f0ccaf9ab None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:50.509976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:50.511116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:50.512255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:50.519110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 265/543] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:50.522698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.545404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:50.546252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.548087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.548524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.550669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:50.552197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:50.575302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 279/544] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:29:50 2022] POST /identity/v3/services => generated 240 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:29:50.579594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.654490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:50.761871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad3dd8-b491-4510-873a-8f30e8faf948 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.762634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e87428-c13a-49b5-a303-10206cd65794 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.798498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:50.819228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:50.819872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.820147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.820416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.821253 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:50.821558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:50.823155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:50.824297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:50.825053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:50.825335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:50.825609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:50.839376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:50.844137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 266/545] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Oct 11 06:29:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:50.848296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:50.849207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:50.850033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 280/546] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:50.851910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.854965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:50.966588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d590f4-1c05-4604-9cc9-af97be0f835e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.971864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64a766-d704-45fe-94e0-3d7c2537c87e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:50.995440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:51.011061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.012311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.012798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.013264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.014363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:51.014993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.015582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.016221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.016511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.016785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.017578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.027936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 281/547] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:51.031180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.035612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.035755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:51.050111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:51.051244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:51.052347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:51.052347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:51.052347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:51.058295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 267/548] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:51.059684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.139873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:51.171466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b4136-e870-4533-b9e4-81ada0664327 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.198288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47516cc-d19a-43db-8aa8-b410d77d6c9b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.213659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.214474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.214905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.215487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.216589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:51.217049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.218549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bbd20-fe2c-4232-8818-820b7fb1d2d6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.232848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 268/549] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:51.237946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1924eff-da03-4307-af87-b8aba8332ad3 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.237946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1924eff-da03-4307-af87-b8aba8332ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.237946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1924eff-da03-4307-af87-b8aba8332ad3 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.239221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 269/550] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:51.240619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1924eff-da03-4307-af87-b8aba8332ad3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.244949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.245404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.245681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.245944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.260942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.263242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.275181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-12c530c5-e14e-4c3c-ab7c-390002154498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:51.288175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:51.289262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:51.290340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:51.290340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:51.290340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:51.290340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 282/551] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:51.292314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.292655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.292993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.316689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:51.318172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:51.365802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1924eff-da03-4307-af87-b8aba8332ad3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.406646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.407086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.408856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.409129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.420472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.422535 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.422648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 283/552] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:51.427526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:51.448426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:51.449542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:51.450615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:51.453809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 270/553] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:51.460965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.473876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:51.615806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb36e6-f399-478c-9896-98e75340d6d9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.629093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:51.655196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b7d08-2ca6-414f-b9c0-aa84f3aee24e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.657493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.658549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.658966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.659484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.661185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:51.661659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.663176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.676862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 284/554] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:51.682680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.718996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.719658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.719936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.720203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.721531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:51.722149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.723646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.750182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 271/555] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:51.754978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.819579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:51.840890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d85855-2952-400f-93aa-1f8f457e3f1b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.870028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e7dcc-e666-4e77-bafb-bd9721da11dc None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:51.884804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.885466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.885972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.886249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.896840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.898998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.909534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b640696-7084-410f-8409-94b83005f074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:51.910321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b640696-7084-410f-8409-94b83005f074 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:51.910703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b640696-7084-410f-8409-94b83005f074 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:51.911077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b640696-7084-410f-8409-94b83005f074 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/498356fa9eaa41e09da95e0a151105dd/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:51.912632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:51.913709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:51.914695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 285/556] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:51.915515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b640696-7084-410f-8409-94b83005f074 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=498356fa9eaa41e09da95e0a151105dd, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:51.915605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.917434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b640696-7084-410f-8409-94b83005f074 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:51.936489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 272/557] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:51 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/498356fa9eaa41e09da95e0a151105dd/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:29:51.937388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b640696-7084-410f-8409-94b83005f074 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:51.975575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.118658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b640696-7084-410f-8409-94b83005f074 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.144118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9335c-7005-4fcb-97af-be6c007a9808 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.159955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.161066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.161347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.161899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.162642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:52.163050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.164604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.182634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.183272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.183583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.183856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.185286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 273/558] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:52.190969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.195935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:52.201363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:52.202285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:52.203080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:52.206233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 286/559] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:52.209342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.213444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.319955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.328715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17e959-e620-4f76-9716-1311bf6f2858 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.370874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.371132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3865a-f74e-4f2a-8eb4-788826dd1559 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.371507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.371785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.372056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.372819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:52.373221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.374599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb34820-7812-4ac2-8556-c019bb0ac452 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.388410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 287/560] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:52.389696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226a3ac-f1cd-4bb9-9c61-275de54da1ca None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.389985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226a3ac-f1cd-4bb9-9c61-275de54da1ca None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.390265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226a3ac-f1cd-4bb9-9c61-275de54da1ca None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.391750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 288/561] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:52.392482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226a3ac-f1cd-4bb9-9c61-275de54da1ca None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.418822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.419505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.419791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.420064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.433202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.434681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-05896517-4f38-4d91-a4b1-0eb589706cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:52.453761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:52.454716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:52.455683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:52.455683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:52.455683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:52.455850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 274/562] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:52.457136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.457419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.457698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.478889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:52.480245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:52.483126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.530847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226a3ac-f1cd-4bb9-9c61-275de54da1ca None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.564229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 275/563] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:52.565544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.576463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.577104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.577378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.577648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.594795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.596350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:52.612233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:52.613181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:52.614035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:52.614035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:52.614035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:52.614211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 289/564] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:52.619615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.684098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.740586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bac5-eff6-4fe0-be56-944e5c0e7047 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.775889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.776506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.776790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.777042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.777794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:52.778106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.779553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.793003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 276/565] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:52.798277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.825364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034af12-b952-4478-8139-d3431bf53c72 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.865843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.897697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.898530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.898905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.899248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.900038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:52.900510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.902153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.907814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cfe96-ef37-490a-84f6-8f40279917ab None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:52.921681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 290/566] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:52.922653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:52.923281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:52.953034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:52.954098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:52.954571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:52.955084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:52.963368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:52.965145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:52.979513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:52.980469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:52.981326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:52.983970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 277/567] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:52.985982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.043591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86963d-3e55-4e95-8244-d28aedfb5092 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.086372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.087016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.087297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.087677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.088263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.089865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:53.103007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:53.104230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:53.105077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 291/568] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:53.105922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.121648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0f8c-f584-4bfd-bb73-f1ce73960059 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.154664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:53.161334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.161965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.162245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.162517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.170007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.171456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-cd1d9d5b-5cf5-4e00-bdad-4ffb548824c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:53.182329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:53.183379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 278/569] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Oct 11 06:29:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:53.184591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e272a8-5619-4304-819e-dacbefcdb079 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.185239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e272a8-5619-4304-819e-dacbefcdb079 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.185239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e272a8-5619-4304-819e-dacbefcdb079 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.186359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 279/570] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:53.190879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e272a8-5619-4304-819e-dacbefcdb079 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.222168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a376f-8939-4718-b97e-9b2f638ee563 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.223769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:53.256831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.257642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.258074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.258338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.259025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:53.259326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.260901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.275454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 292/571] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:53.275727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.309112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e272a8-5619-4304-819e-dacbefcdb079 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.346443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.347052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.347327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.347632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.348242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:53.348544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.349886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067de98-b304-432e-aa91-ab0117d42c1b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.359067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 280/572] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:53.360092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.360365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.360640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.381736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:53.382932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:53.386217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5804b6-20a0-47ce-8034-bf7c04d34d28 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.387875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:53.425865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.426479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.426760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.427038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.427785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:53.428087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.429540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.447592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 293/573] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:53.448393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.468099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 281/574] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:53.468949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.493830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:53.578309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1317a58-e38a-46b2-93d1-ce966257ad86 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.590220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f9f2-7239-4dc3-b713-b9528667818e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.616542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.617423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.617767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.618110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/b0080cb54dc84d2da383b5d97bd64fa7/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.633311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=b0080cb54dc84d2da383b5d97bd64fa7, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.633698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.634305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.634585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.634853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.635193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.635504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:53.635808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.637376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.647587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 282/575] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:53.649644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.656322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 294/576] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:29:53 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/b0080cb54dc84d2da383b5d97bd64fa7/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:29:53.657286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.675287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:53.765432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:53.797474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0fad4-2c32-4447-ad2e-562b47c5e482 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.819440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21de68-6464-423e-ad73-3a99feeeffa9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:53.843263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.844204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.844487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.845204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.864705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:53.865334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:53.865660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:53.865923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:53.866649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] RBAC: Adding query filter params (interface=public, service_id=8687688390f947e68cc83dabb61a9613, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:53.866997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.868503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.877038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 295/577] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/endpoints?service_id=8687688390f947e68cc83dabb61a9613&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:53.882249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:53.939047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:53.940642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:53.990396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b3032-8dc8-4db4-983c-94671951905f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.003799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.027492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.028461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.028961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.029434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] PATH_INFO: `/v3/services/network` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.030317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.032167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:54.042960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:54.044146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:54.045029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:54.048051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 296/578] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:54.049187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.057673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.074307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 283/579] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:29:54.075170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.164108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a11d3d-2062-4850-8396-eb5376d61f5f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.202349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cca68-fb96-4235-9cb0-9c47f41ac97a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.212169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.212777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.213157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.213429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.214047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.215570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.221221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 297/580] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:54.228419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.232871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.255874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.257265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.257539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.258042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.258706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] RBAC: Adding query filter params (name=network) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:54.259005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.263371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.275292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 284/581] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:54.283270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.398238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.431577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66730-5fc3-4402-9023-1228456ca989 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.467799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fe72-2baf-487c-ac5c-2a5a90107776 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.474911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.475582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.475862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.476131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.490330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.491815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-15b47691-e8bc-4dfa-bf98-a6b85b20deb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:54.507904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:54.508885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 298/582] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02233a0-3748-46d8-afc7-bc66aa5b8356 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02233a0-3748-46d8-afc7-bc66aa5b8356 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.510671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02233a0-3748-46d8-afc7-bc66aa5b8356 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.512193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 299/583] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:54.516866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02233a0-3748-46d8-afc7-bc66aa5b8356 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.517313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.518506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.518788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.519060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.519774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] RBAC: Adding query filter params (type=network) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:54.520079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.521451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0d953-1d1b-45f8-a884-177c8ea82a66 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.527688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 285/584] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:54.528897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.529185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.529461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.553140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:54.554258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:54.578434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.628374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 286/585] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:54.635585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.637606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02233a0-3748-46d8-afc7-bc66aa5b8356 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.676466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.677138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.677422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.677703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.678486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:54.678891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.680438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.696141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 300/586] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:54.703047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.724813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.788793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1878e-085c-4376-b4be-c0b7bd0a0a64 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.855807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3c24-9a82-4c0e-aeef-277a2f76427c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:54.873230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.874193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.874674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.875137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.876077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] RBAC: Adding query filter params (interface=public, service_id=8360171f96314940bbcbd466437819e8, region_id=RegionOne) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:54.876627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.879922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.889514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:54.899218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 287/587] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/endpoints?service_id=8360171f96314940bbcbd466437819e8&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:54.905341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:54.935693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:54.936336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:54.936624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:54.936894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:54.953740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:54.955342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:54.977246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:54.978218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 301/588] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:54.979209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.043859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7267d-5081-4d56-bdab-95858478899e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.058357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:55.086200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.086810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.087088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.087362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.088688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:55.090108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:55.098607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40ff8b-858c-44c0-83cb-cae5dab5e786 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.099822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 288/589] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:55.101051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.179785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.181162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.181873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.183484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.207605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:55.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:55.217089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:29:55.217906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:55.224193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 302/590] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:55.243510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.294882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5c80-9517-4cad-9890-18499f3708ac None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.296958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:55.338820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.339483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.339767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.340038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.340740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:55.341046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:55.342507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:55.367062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 289/591] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:55.376404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.388718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500395ae-72e9-4b2f-9fbe-ccacb65c303f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.431400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:55.439787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.441203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.441705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.442206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.443884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:55.444372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:55.446680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:55.459386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 303/592] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:29:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:55.471523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.578669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:55.586676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202989d-f3ef-408b-999b-9fdbb6eca2c9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.591768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691b189-9d75-4de0-861c-37823939b1f9 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.625100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.625727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.626002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.626271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.638534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:55.640083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:55.657535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:55.659373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:55.660284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:55.660284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:55.660284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:55.663738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 304/593] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:55.665686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.700180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.700713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.700995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.701263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.701990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=fbf7156a9d9448a6987fe7999543cedb) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:55.702292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:55.703734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:55.721860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 290/594] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:55 2022] GET /identity/v3/role_assignments?user.id=fbf7156a9d9448a6987fe7999543cedb&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:55.806131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67346d94-5017-4a20-a568-db162d923f3c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:55.860133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:55.860819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:55.861144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:55.861502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:55.862175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:55.862474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:55.863945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:55.877471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 305/595] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:55.883304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:55.992479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:55.996567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431ad35-a71b-4fd5-9512-a7f2e4af1ed5 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:56.030674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:56.031306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.031616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.031887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.040354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:56.041806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-376ddc09-02a5-495c-9950-371eb03808bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:56.053731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:56.054687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 06:29:56.055632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:56.059575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 291/596] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:29:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:56.060998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:56.072235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:56.078389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d143b-584e-41fb-8630-553586292dad None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.078675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d143b-584e-41fb-8630-553586292dad None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.078923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d143b-584e-41fb-8630-553586292dad None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.081233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 292/597] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:29:56.088592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.088920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.089197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.114772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:56.116342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:56.183236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93646309-35f1-4adf-aea2-55ad0286a928 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:56.206694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 293/598] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:56.242125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:56.242894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.243121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.243392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.244148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:56.244445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:56.245874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:56.278041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 306/599] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Oct 11 06:29:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:56.284908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:56.390802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:56.413372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95d23-9264-4d02-b749-0fff50f5b46c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:56.458817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:56.459445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.459724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.459992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.532470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:56.534659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:56.610779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:56.627792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:56.638778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 294/600] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:29:56.640497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:56.743483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66106f8c-a962-45a9-bf1d-407944894bba None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:56.771343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:56.818960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:56.819532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.819815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.820085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.837204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:56.838645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:56.840847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a13d-1f2e-4f4e-a2b3-e0e50df28ed3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-96f3f4e7-c825-4886-9e55-c2751d17ad64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:56.868385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:56.869392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:56.870252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:56.870252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:56.870252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:56.874018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 295/601] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:56.875269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73909373-156e-43ec-b7db-6fb2791a7e67 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.875592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73909373-156e-43ec-b7db-6fb2791a7e67 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.876090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73909373-156e-43ec-b7db-6fb2791a7e67 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.877734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 296/602] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:56.878418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73909373-156e-43ec-b7db-6fb2791a7e67 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:56.893647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad9243-b919-45ed-8a8c-a8f00ed269cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:56.894590 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad9243-b919-45ed-8a8c-a8f00ed269cd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.894871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad9243-b919-45ed-8a8c-a8f00ed269cd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.895138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad9243-b919-45ed-8a8c-a8f00ed269cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.909600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad9243-b919-45ed-8a8c-a8f00ed269cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-67ad9243-b919-45ed-8a8c-a8f00ed269cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:29:56.916348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:56.917243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:29:56.918055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:56.918912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 307/603] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:29:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:56.920113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:56.920395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:56.920668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:56.926416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:56.946053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:56.947544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:56.988864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73909373-156e-43ec-b7db-6fb2791a7e67 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.025517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.026262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.026631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.027000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.027985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:57.028396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.030475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.040072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 308/604] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:57.042195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.046584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 297/605] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:29:57.062611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.148193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:57.201576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b42bd-90d8-424d-a00b-55d8aa5b53c4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.256546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.257168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.257437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.257702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.258361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:57.258652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.260167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.277116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d920c78-aafd-4ae7-8c06-db844e3b6667 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.278733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 309/606] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:57.284365 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.311782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:57.340704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254a46-1010-4246-9453-18493af96703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.342256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254a46-1010-4246-9453-18493af96703 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.342537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254a46-1010-4246-9453-18493af96703 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.342812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254a46-1010-4246-9453-18493af96703 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.356599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c254a46-1010-4246-9453-18493af96703 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.358061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c254a46-1010-4246-9453-18493af96703 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-1c254a46-1010-4246-9453-18493af96703 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:57.370674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:57.371830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:57.372687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 298/607] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:57.373150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254a46-1010-4246-9453-18493af96703 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.429867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c91b-ca4d-4283-a398-ef69e69993fe None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.450146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:57.474250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.475471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.475872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.476245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.489792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254a46-1010-4246-9453-18493af96703 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.490578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.493699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:57.515012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:57.516122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:57.518509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:57.518509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:57.518509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:57.518509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 310/608] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:29:57.518509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.539442 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.540207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.540632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.541007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.541667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.543555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:57.555405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:57.557417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 06:29:57.559200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:57.565500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 299/609] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:57.565500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.597937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:57.637648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d02c2-b94d-40ec-9d6b-a7d285ea8329 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.672578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.673102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.673374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.674419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.674855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:57.675154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.676625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.683764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba39743-7f26-4b0c-adb4-09c8930b3b42 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.689254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 311/610] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:57.690567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.721070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.721748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.722155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.722465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.723144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:57.723618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.725130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.725835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:57.738247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 300/611] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:57.739062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.802134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080be95-658e-4c81-ba5e-cf2d9a7a2bdb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.829145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:57.839986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.840906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.841397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.841882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.842667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:57.843138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.844654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc54b4-703a-4a8e-9db3-656a5eff76fe None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:57.844824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fc3e0-bdc9-4214-aad3-9286c8d212c9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.850423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 312/612] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:57.852192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74eca9-8916-4fe9-ab53-d3c7afa02dc3 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.852634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74eca9-8916-4fe9-ab53-d3c7afa02dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.853038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74eca9-8916-4fe9-ab53-d3c7afa02dc3 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.859211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 313/613] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:57.860150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74eca9-8916-4fe9-ab53-d3c7afa02dc3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:57.879460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:57.880053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:57.880328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:57.880588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:57.881302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=498356fa9eaa41e09da95e0a151105dd) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:57.881606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:57.883026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:57.888660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 301/614] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/role_assignments?user.id=498356fa9eaa41e09da95e0a151105dd&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:57.889421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.009084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:58.021269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74eca9-8916-4fe9-ab53-d3c7afa02dc3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.021858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8f16a-945c-4949-b28f-5b18558a00e4 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.065446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:58.066493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.067026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.067522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.067894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:58.068531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.068806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.069043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:58.069145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.069623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:58.071514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1330c-c378-4ca0-8707-2d30d26d10a1 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:58.078994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:58.079493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 302/615] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:58.081053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:58.081740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.082239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.082744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:29:58.094770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:58.095870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:29:58.096834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:58.097375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 314/616] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:29:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:58.098207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.104385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:58.105846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:58.116702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:58.191224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 303/617] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:58.192466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.252992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4e200-8c8a-445e-add2-2adcd566c334 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.308215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:58.309177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.309675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.310172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.311055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:58.312649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:58.341380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abe733-d39e-4a7c-9e13-65e2970a4394 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.356393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 315/618] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:29:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:58.399324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:58.400017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.400348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.403687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.403687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:58.403687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:58.403687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f296c93-e3dc-4dc5-8e1c-45e8ee24647f None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:58.414022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:58.428490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 304/619] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Oct 11 06:29:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:58.435591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.453628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:58.471785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6081ab-bfc1-4c90-83c6-2948682aa755 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.472438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6081ab-bfc1-4c90-83c6-2948682aa755 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.472937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6081ab-bfc1-4c90-83c6-2948682aa755 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.474872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 305/620] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:29:58.485322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.486552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.487032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.524699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:29:58.526058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:29:58.565444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d2204-3b95-46cb-b317-ea10a9cf7162 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.602542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:58.603168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.603453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.603729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.648073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 306/621] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:29:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:29:58.649040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.680128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:58.681592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:58.694446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:58.753583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3856e2-027f-435b-992e-d6d828b0daac None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.794770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:58.811014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:58.811672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:58.811945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:58.812214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] PATH_INFO: `/v3/services/network` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:58.812802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:58.820225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:58.833434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:58.834433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 06:29:58.835242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:58.837577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 316/622] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:29:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:29:58.838815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 307/623] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:58 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:58.842714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.846749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:58.991681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72406b-d6b2-4922-8888-1ae3aae5aa3e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:58.997917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5b159-1398-4493-bb8d-283c6b89842d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.009912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:59.032836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.033577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.033927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.034375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.040682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.041548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.042571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.043060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.044430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:59.044934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.046328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.047098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.048078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.058824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 317/624] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:29:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:59.060190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:29:59.068307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:59.069313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:29:59.070204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:29:59.070204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:29:59.070204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:29:59.070204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 308/625] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:59.071243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.123691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:59.193432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f52b-fd3f-4531-898a-7013dfb57193 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.204000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcaeeb2-b221-4a61-89b7-78638c4348d2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.230561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.231290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.231629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.231978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.232708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] RBAC: Adding query filter params (type=network) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:59.233224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.234927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.245380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 309/626] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:59.245380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.247381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.248300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.248778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.249247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.250050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.252216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:59.261610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:59.262974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:29:59.263891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:59.266765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 318/627] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:59.267864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.309839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:59.359199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea840b-f93d-4c8b-bc61-68f40fed69dd None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.367484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:59.374554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f4f89-1d2b-4d8d-aa78-63e8a3eeed6e None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.401847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.402471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.402808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.403085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.403785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:59.404092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.405581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.415512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.416408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.416911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.417384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.418257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.419864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.419977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 310/628] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:59.420824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.443746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 319/629] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:29:59 2022] POST /identity/v3/endpoints => generated 337 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:29:59.452552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.537043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b01d1a-bf0c-40e5-89fd-b2b0ace0d6dc None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.594422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.595058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.595973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.596256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.596908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:59.597210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.598718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.611778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6bd0f-7e4e-47bb-bfe5-a898d1740ca2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.612660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 311/630] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:59.614299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:59.619276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.650541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.651611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.652138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.652647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.661094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.662867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:29:59.675309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:29:59.676377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:29:59.677269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:29:59.678278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:29:59.680192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 320/631] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:29:59.688637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.818071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe35376-cad4-44f9-96df-b8ff5dd39b83 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.819224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaf7f0-4709-4e98-81f0-79902418a4c7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:29:59.856069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.856681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.856959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.857231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.857859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:59.858161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.859022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:29:59.859613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.860096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:29:59.860623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:29:59.861180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:29:59.862223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:29:59.862708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:29:59.864349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89283828-62ed-4be8-b0bd-efb505e6dd86 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:29:59.864812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 312/632] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:59.878511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 321/633] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:29:59.884241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:29:59.993717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:00.004677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f8c2d-5ef6-4625-997b-09c59b5fd16d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:00.046737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:00.047725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:00.048188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:00.049217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:00.050180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=b0080cb54dc84d2da383b5d97bd64fa7) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:00.050732 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:00.052393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9788251-c5c7-4f6b-8e91-50f12531e9df None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:00.059248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 313/634] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:29:59 2022] GET /identity/v3/role_assignments?user.id=b0080cb54dc84d2da383b5d97bd64fa7&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:00.439676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dbd8e-e532-4e36-9724-ab42f239aab5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:00.440625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dbd8e-e532-4e36-9724-ab42f239aab5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:00.441128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dbd8e-e532-4e36-9724-ab42f239aab5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:00.442946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 322/635] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:00.448777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aad6b5-2c57-45b5-9253-463af2216666 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:00.449296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aad6b5-2c57-45b5-9253-463af2216666 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:00.449795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aad6b5-2c57-45b5-9253-463af2216666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:00.468950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e6aad6b5-2c57-45b5-9253-463af2216666 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:00.470515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e6aad6b5-2c57-45b5-9253-463af2216666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:00.543266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 314/636] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:00.762092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddcb50-5fbc-432d-8065-79eb1f2b7beb None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:00.762263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddcb50-5fbc-432d-8065-79eb1f2b7beb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:00.762538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddcb50-5fbc-432d-8065-79eb1f2b7beb None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:00.764219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 323/637] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:00.769400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:00.769690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:00.769965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:00.788366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:00.789539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:00.815071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddcb50-5fbc-432d-8065-79eb1f2b7beb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:00.838732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:00.863502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 315/638] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:00.943723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddcb50-5fbc-432d-8065-79eb1f2b7beb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:00.979063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:00.979734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:00.980008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:00.980278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:00.980862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:00.982229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:00.992041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:00.993004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:00.994245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 324/639] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:01.001408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:01.017659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:01.104048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42e9d9-dd8e-42ec-8284-fb4b1030d6eb None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:01.139346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:01.140322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:01.140832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:01.141303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:01.143073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:01.143615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:01.143961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:01.145425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:01.155409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 316/640] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:01.156602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:01.235406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:01.260794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe73f1-1ae8-41df-b463-b13be8d2f094 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:01.264293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e201bc-9a3a-4fbe-a07d-9e8aa2769de4 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:01.295106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:01.300250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:01.301083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:01.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:01.301682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:01.301982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:01.302245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:01.302352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:01.302516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:01.303153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:01.303332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:01.303479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:01.304847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:01.305226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:01.315606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 317/641] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:01.316311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:01.317634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:01.319203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:01.322829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 325/642] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:01.331474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:01.444281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16bfb-0432-4a1e-9fc8-fff94fbd31b6 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:01.480448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:01.481179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:01.481457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:01.481725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:01.482364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:01.482662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:01.484073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc3f69-8e14-4073-a22e-bccdf92984e6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:01.489975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 318/643] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:01.495601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:01.599171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bb842-1bae-418a-9a75-b1cd5c3294d4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:01.633810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:01.634458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:01.634736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:01.635003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:01.635652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:01.635946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:01.637300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c30da-f4b9-4f0e-a83e-e9068c2b9bcf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:01.643227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 326/644] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:02.217816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218732e-0c26-4fd6-a0c1-22772d7a629a None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:02.218318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218732e-0c26-4fd6-a0c1-22772d7a629a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:02.218396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218732e-0c26-4fd6-a0c1-22772d7a629a None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:02.219742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 319/645] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:02.225944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:02.226226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:02.226500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:02.243548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:02.250443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:02.251801 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:02.301907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ac61-d3cf-45af-9c40-61eb252e7f72 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:02.302217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ac61-d3cf-45af-9c40-61eb252e7f72 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:02.302454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9ac61-d3cf-45af-9c40-61eb252e7f72 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:02.304116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 320/646] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:02.310890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:02.311211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:02.311448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:02.334140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:02.344487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:02.345719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:02.353313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 327/647] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:02.422269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 321/648] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:02.679348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:02.711773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:02.855444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd34f-d8b3-4978-9069-4253c4dd922b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:02.892486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2318162-38be-452d-851b-22f5e61006d1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:02.904180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:02.904791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:02.907437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:02.907437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:02.907437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:02.907979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:02.912394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:02.927610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 328/649] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:02 2022] POST /identity/v3/services => generated 269 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:02.931199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:02.937195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:02.937847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:02.938120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:02.938386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:02.958296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:02.963867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:02.964880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:02.965683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:02.968144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 322/650] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:30:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:02.974180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:03.091783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c8ff7-018c-4b8e-8a4b-ad6ca739ba5d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:03.128329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:03.128941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.129219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.129491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.130156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:03.130453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:03.131864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:03.140385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 329/651] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Oct 11 06:30:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:03.146352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:03.252392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147700cc-fd10-47bf-adc1-657c01b36a0e None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:03.291440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:03.292402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.292858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.293306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.304147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:03.305894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:03.321826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:03.322776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:03.323712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:03.323712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:03.323712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:03.325370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 323/652] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:03.335462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:03.392541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:03.438233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44ea56-73d2-420b-8341-25a343dd3a25 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:03.475965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:03.476578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.476857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.477135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.477846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:03.478154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:03.479562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:03.495787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 330/653] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:03.498186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:03.550773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:03.611436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e2bd7-5e1a-498c-8cc5-9a2b9a7ba8d1 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:03.648563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:03.649542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.650000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.650445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.659442 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:03.661106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-3408877d-c08d-4a4c-987f-20501a8988ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:30:03.672775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:03.673736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:30:03.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:03.676759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 324/654] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:30:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:03.684533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:03.725754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0bcf6-6e7c-4071-817e-ed1a804aa2c9 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.726105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0bcf6-6e7c-4071-817e-ed1a804aa2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.726313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0bcf6-6e7c-4071-817e-ed1a804aa2c9 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.727917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 325/655] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:30:03.735844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.736252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.736630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.756352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:03.757715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:03.789920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbc226-6a41-4e02-ab3a-dccab95d2ba4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:03.812652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:03.839551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:03.840520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.840859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.841205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.842017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:03.842409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:03.843737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 326/656] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:03.847613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1a3a4-4db0-4859-b391-b83f6d94264f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:03.865995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 331/657] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Oct 11 06:30:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:03.872488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:03.907003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:03.930912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446b105-c58a-4968-9ab6-82aa50653e92 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.931238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446b105-c58a-4968-9ab6-82aa50653e92 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.931495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446b105-c58a-4968-9ab6-82aa50653e92 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.933598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 332/658] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:30:03.939821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:03.940099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:03.940371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:03.960091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:03.961380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:03.983935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd00a2-fd8a-4b3a-80ee-7a6d88346b05 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.018401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.019030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.019303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.019599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.029581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.031021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.039900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 333/659] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:04.045729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:04.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:04.047960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:04.047960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:04.047960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:04.047960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 327/660] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:04.053756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:04.138100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:04.159383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1883ca-a7b5-4754-80e4-5c35ee7bd796 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.166681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:04.205491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.206103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.206382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.206652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.207295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:04.207781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.209657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.225683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 334/661] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:04.229381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:04.231670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:04.247251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55382-abab-4710-97c7-77295a3017aa None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.286405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.287072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.287281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.287588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.288700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.290323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.315467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 328/662] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:04 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:30:04.317441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:04.344238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b400510-02d4-4b2b-996a-b05ea6aa23ad None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.389905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.390454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.390745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.390998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.399020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.400540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:30:04.415479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:04.416402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:04.417272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 335/663] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:04.420856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:04.452479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a077cf7-c84b-40b2-827a-f1dff04bbbd8 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.455815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:04.489446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.490072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.490356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.490628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.491292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.492720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.508732 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 329/664] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:04 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:30:04.551342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c1dda-11d8-4e2c-8c47-4a928b0c0e7a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.597429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.598051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.598338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.598613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.599319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:04.599657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.601141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b5146-2833-468b-93d6-74b9df7cc30e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.619562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 336/665] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:04.621832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:04.631984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:04.735765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89435abe-bf67-4677-976a-57498a0b17ab None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:04.771490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43776f40-d8d3-4ba8-a9c9-9eb843fa6fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:04.772199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43776f40-d8d3-4ba8-a9c9-9eb843fa6fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:04.772487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43776f40-d8d3-4ba8-a9c9-9eb843fa6fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:04.772761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43776f40-d8d3-4ba8-a9c9-9eb843fa6fa3 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/b0080cb54dc84d2da383b5d97bd64fa7/roles/be335f32244b4b4991452bafc82027f5` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:04.774947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43776f40-d8d3-4ba8-a9c9-9eb843fa6fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=b0080cb54dc84d2da383b5d97bd64fa7, role_id=be335f32244b4b4991452bafc82027f5)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:04.776423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43776f40-d8d3-4ba8-a9c9-9eb843fa6fa3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:04.795575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 330/666] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:30:04 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/b0080cb54dc84d2da383b5d97bd64fa7/roles/be335f32244b4b4991452bafc82027f5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:30:05.303080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8c118-0d4b-436a-9ff7-d20b9bda1aa7 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:05.303744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8c118-0d4b-436a-9ff7-d20b9bda1aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:05.303744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8c118-0d4b-436a-9ff7-d20b9bda1aa7 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:05.305566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 337/667] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:05.311317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:05.311871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:05.311964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:05.331437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:05.332624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:05.399005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:05.409123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 331/668] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:05.680815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8c118-0d4b-436a-9ff7-d20b9bda1aa7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:05.764166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:05.789471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8c118-0d4b-436a-9ff7-d20b9bda1aa7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:05.824633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:05.825360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:05.825644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:05.825917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:05.826549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:05.828005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:05.837609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:05.838537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:05.839449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 338/669] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Oct 11 06:30:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:05.844864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:05.948575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc62d1-c208-48db-a54a-a85cbd6ea83f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:05.982641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:05.983249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:05.983558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:05.983828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:05.984457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:05.984757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:05.986657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:05.991721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 332/670] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:06.000743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:06.103852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53fb7-30fa-4ce6-9bf8-c47001015625 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:06.141787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:06.142761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.143257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.143788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.144645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:06.145140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:06.146772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:06.152572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 339/671] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:06.160991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:06.218329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:06.259931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adee52-fbd1-4317-9cfe-1387626332db None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:06.297692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:06.298866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.299147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.299431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.300146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] RBAC: Adding query filter params (interface=public, service_id=e31944776e014aee8d59acd40974ad74, region_id=RegionOne) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:06.300454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:06.301809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc36d38-9c74-4f9a-a99c-1c2730e98629 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:06.307548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 333/672] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:30:06 2022] GET /identity/v3/endpoints?service_id=e31944776e014aee8d59acd40974ad74&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:06.312904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:06.322873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:06.420020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053a48-3769-475d-9a91-4e49c81ab6d9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:06.455743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6758f-0557-4df4-ae3f-3b30ac1148d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:06.456354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6758f-0557-4df4-ae3f-3b30ac1148d6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.456632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6758f-0557-4df4-ae3f-3b30ac1148d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.456906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6758f-0557-4df4-ae3f-3b30ac1148d6 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.457510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6758f-0557-4df4-ae3f-3b30ac1148d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:06.458870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6758f-0557-4df4-ae3f-3b30ac1148d6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:06.465483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 340/673] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:30:06 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:30:06.733751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eadb3c-1e19-4c46-b261-d8e6c2583895 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.734270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eadb3c-1e19-4c46-b261-d8e6c2583895 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.734270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eadb3c-1e19-4c46-b261-d8e6c2583895 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.735818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 334/674] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:06.735995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b5c35-ddc9-45d2-ad56-f2d4933f82a9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.736226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b5c35-ddc9-45d2-ad56-f2d4933f82a9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.736497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b5c35-ddc9-45d2-ad56-f2d4933f82a9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.737968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 341/675] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:06.743980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684f491-dea4-472f-8573-3af4d313a925 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.743980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684f491-dea4-472f-8573-3af4d313a925 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.743980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684f491-dea4-472f-8573-3af4d313a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.745175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7eb8-a03a-4cef-b554-3a8f8496d79e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:06.745463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7eb8-a03a-4cef-b554-3a8f8496d79e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:06.745739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7eb8-a03a-4cef-b554-3a8f8496d79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:06.761760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e684f491-dea4-472f-8573-3af4d313a925 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:06.762965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e684f491-dea4-472f-8573-3af4d313a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:06.769464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-446b7eb8-a03a-4cef-b554-3a8f8496d79e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:06.770844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-446b7eb8-a03a-4cef-b554-3a8f8496d79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:06.833523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 335/676] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:06.843863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:06.846064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 342/677] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:07.076482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed0ce-349f-40f5-a865-fae034f413ca None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.077146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed0ce-349f-40f5-a865-fae034f413ca None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.077640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed0ce-349f-40f5-a865-fae034f413ca None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.079468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 336/678] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:07.085422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.085916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.086412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.103791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed0ce-349f-40f5-a865-fae034f413ca None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.105647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:07.106994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:07.162292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.182877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 343/679] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:07.183950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.218051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed0ce-349f-40f5-a865-fae034f413ca None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.259636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.260602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.261099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.261621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.262487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.264068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:07.275787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:07.276764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:07.277621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:07.285512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 337/680] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:07.287101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.308514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaeae2-f05b-4bd7-b726-ee9540093a4e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.338665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.348239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.349096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.349524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.350485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.351255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.353327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:07.367102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:07.368493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:07.369531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:07.373073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 344/681] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:07.382296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.408168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff109ff-5d0f-43f2-980f-9ad8f6f190ea None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.427484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.448360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.449292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.449578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.449945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.450584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:07.450887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.452290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.459075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 338/682] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:07.463113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.508299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531f991-31d8-4d8d-b227-9d5de9a8dc11 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.542051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.542649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.542923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.543190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.543858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:07.544154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.545510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.551625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 345/683] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:07.551868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.579308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d87ae-5910-4d29-9107-dccd8f62ea95 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.613287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.613918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.614343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.614616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.618862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.625725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:07.633919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:07.634850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 339/684] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:07.640332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.665008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc17d9-614f-4bbe-acd6-17fcee442e80 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.683270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.711182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.712341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.712341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.712512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.713110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:07.713407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.714770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.725101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 346/685] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:07.725101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.749104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5d2f-ef55-4bb0-8f09-61b02ef25040 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.782213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.783489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.783774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.784052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.784684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:07.784985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.786353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.790991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 340/686] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:07.791836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.833665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9b3a9-a513-42c7-9571-bac81be79fe3 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.855754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.868544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.869085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.869366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.869643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.870305 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:07.870606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.872017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.880106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 347/687] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:07.882042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:07.902403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95746e3-76ff-4cd7-a398-1e6a925ca0d2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:07.910315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:07.943807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:07.944815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:07.945109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:07.945390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:07.947112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] RBAC: Adding query filter params (interface=public, service_id=be895ec9dda14edd9cde38254933570a, region_id=RegionOne) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:07.947112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:07.948107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:07.953790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 341/688] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/endpoints?service_id=be895ec9dda14edd9cde38254933570a&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:07.954614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:08.011352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbcb27-cab4-422e-be9d-b09da16e66e9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.051319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.051964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.052242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.052509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.053209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] RBAC: Adding query filter params (interface=public, service_id=1dfd9690103e47f29e7d6868e5f64038, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:08.053514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.054875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.059901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 348/689] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/endpoints?service_id=1dfd9690103e47f29e7d6868e5f64038&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:08.060666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:08.076983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe113-87a7-4755-83fd-ec9e280d2856 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.126634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.126821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:08.127329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.127524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.127829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.138727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.140194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:08.155102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:08.156195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:08.157095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:08.157095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:08.157095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:08.158546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 342/690] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:08.159664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:08.182164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75364392-3f16-4c36-9db2-21f9c7f62399 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.186257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:08.223475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.224100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.224373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.224641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.225403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.227341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.247802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 349/691] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:30:08 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:30:08.248604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:08.267369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e4806-1711-4d61-944e-197c99ca7ff2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.313759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.315135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.315615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.316040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.317309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.319211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.334854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 343/692] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:30:08 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:30:08.423618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459069f-a91d-4c9d-be46-09a9907668ef None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.458475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:08.500884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.501554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.501835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.502109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.502771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:08.503077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.504528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.525879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 350/693] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:08.527207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:08.634094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:08.650495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb88638-cb67-45b2-a461-266daad6e827 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.687992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.688556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.688880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.689152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.697949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.699409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-0668437d-d510-4667-bcc3-3f1a1197eeec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:30:08.710780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:08.711813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 06:30:08.712740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:08.715974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 344/694] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:30:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:08.719389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:08.724428 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e912-ae95-4593-bff7-0db709b23d72 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.724714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e912-ae95-4593-bff7-0db709b23d72 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.725075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e912-ae95-4593-bff7-0db709b23d72 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.726677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 345/695] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:08.732026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.732312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.732587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.756974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:08.758311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:08.848248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 346/696] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:08.885761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccf7f5-9916-404e-af0c-faa9f6ff2a79 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:08.923312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:08.923963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:08.924238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:08.924509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:08.925243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:08.925547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:08.926979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:08.950107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 351/697] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Oct 11 06:30:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:08.951688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.039314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.063810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8859004-0af4-4668-8db7-dc2f43aae59c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.098962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.099654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.099934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.100244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.152889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.175234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.177588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.192677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.274820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.284855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42a85-a8d8-4553-a530-0f58a8c25e0a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.313571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 347/698] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:30:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:30:09.315135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.323334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.324258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.324707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.325166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.325911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.327458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:09.339281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:09.340338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 06:30:09.341205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:09.349090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 352/699] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:09.351854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.443002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e993-da5c-4c50-87aa-1aacbf0c9d3c None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.486282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.486902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.488624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.489583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.490300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.494794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d710c70-5b10-420a-8525-496a5c5a5d25 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.502557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.505441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:09.523300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:09.524292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:09.525185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:09.525185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:09.525185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:09.527562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 348/700] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:09.533711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.537836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.539281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.539609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.539877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.540505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:09.540805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.542168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.551495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 353/701] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:09.553832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.585542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.668132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03c8bc-5194-43a3-afb4-b157a884c892 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.674019 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff55bf7-e4d1-4b73-8253-a557ee1496f4 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.704877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.705740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.706405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.706574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.707345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:09.707748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.709261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.714992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.719448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.719585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 354/702] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:09.720782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.721266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.722200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.723219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:09.723557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.724979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.739503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 349/703] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:09.744226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:09.753758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.837942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31856c0-7abb-49fc-ab1b-55f4f5852947 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.856469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:09.872784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.873389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.873665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.873912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752258b-5004-41f4-ac7c-ca953dc6483a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:09.874020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.874584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.875991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.909282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 355/704] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:30:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:09.921846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:09.922909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:09.923505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:09.923984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:09.932305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:09.934211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:30:09.947755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:09.948698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:09.949589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:09.957256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 350/705] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:09.960596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:10.068430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:10.094293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871206-2e39-4f27-96ec-95d8236ba683 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:10.136946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:10.137567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:10.137848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:10.138117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:10.138815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:10.139120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:10.140704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3d8dc-3b96-4679-8ad2-79ca49452527 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:10.154981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 356/706] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:10.161201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:10.273837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2f429-3d15-4693-a3bd-0a87efbfabe5 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:10.308431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:10.309294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:10.309748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:10.310169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:10.311128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] RBAC: Adding query filter params (role.id=be335f32244b4b4991452bafc82027f5, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=b0080cb54dc84d2da383b5d97bd64fa7) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:10.311614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:10.313188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803edaa-5f2f-46a0-80dc-f6be9d9126d2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:10.320325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 351/707] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:30:10 2022] GET /identity/v3/role_assignments?user.id=b0080cb54dc84d2da383b5d97bd64fa7&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=be335f32244b4b4991452bafc82027f5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:10.500653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b984d-3869-4a06-b372-84248f2cd51c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:10.501230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b984d-3869-4a06-b372-84248f2cd51c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:10.501844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b984d-3869-4a06-b372-84248f2cd51c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:10.503899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 357/708] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:10.510675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89729def-c726-48b6-a13c-ac3028045fe0 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:10.510675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89729def-c726-48b6-a13c-ac3028045fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:10.510675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89729def-c726-48b6-a13c-ac3028045fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:10.544446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-89729def-c726-48b6-a13c-ac3028045fe0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:10.545988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-89729def-c726-48b6-a13c-ac3028045fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:10.549212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:10.680934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 352/709] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:10.703446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08aa464-80d8-42f0-8aa3-1e1c9102ee80 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:10.703446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08aa464-80d8-42f0-8aa3-1e1c9102ee80 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:10.703446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08aa464-80d8-42f0-8aa3-1e1c9102ee80 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:10.707437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 358/710] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:10.711455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:10.711455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:10.711455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:10.732554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:10.733745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:10.817207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 353/711] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:10.974782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08aa464-80d8-42f0-8aa3-1e1c9102ee80 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.118367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.138524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08aa464-80d8-42f0-8aa3-1e1c9102ee80 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.176670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.177675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.178157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.178670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.179779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.181358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:11.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:11.192041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 06:30:11.192862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:11.195847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 359/712] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:11.201875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.203880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:11.206224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:11.252963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186654cd-8028-41ad-ab0f-755e7bef4b87 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.297137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.298633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.299449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.299919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.301037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.302990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:11.315868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:11.316849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:11.317636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:11.320215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 354/713] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:11.325489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.329959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dbe82-29b5-406e-a455-58783f83a731 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.365772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.366385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.366664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.366935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.367602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:11.367906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.369273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.374647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 360/714] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:11.384445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.439768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0e3dc-da92-4b7a-aff9-12960466adc1 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.470498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:11.473114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:11.480165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.481016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.481506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.481987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.482800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:11.483290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.484855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.499392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 355/715] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:11.500566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.510741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f0a67-c6c8-4a7f-8e81-050ff3b9fd42 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.546381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.547003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.547278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.547813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.548763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:11.549064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.550699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.556081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 361/716] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Oct 11 06:30:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:11.561318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.606770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b4ee5-2050-4efe-b3fc-a9790789076a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.645799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.646777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.647250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.647762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.648580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:11.649072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.650606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.660959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 356/717] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:11.662013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:11.676268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76587-eed0-4c3a-b60a-b401452c2cf5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.707400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:11.715087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8293cd-10f4-4ca3-9b79-6a9bd5ddc7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.715742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8293cd-10f4-4ca3-9b79-6a9bd5ddc7a9 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.716023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8293cd-10f4-4ca3-9b79-6a9bd5ddc7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.716299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8293cd-10f4-4ca3-9b79-6a9bd5ddc7a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.716985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8293cd-10f4-4ca3-9b79-6a9bd5ddc7a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.718792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8293cd-10f4-4ca3-9b79-6a9bd5ddc7a9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.727621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:11.755624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 362/718] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:30:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:11.775853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0071-05cb-4eb2-b151-f098740cd572 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:11.822277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b41c6-ab1b-4064-af9f-171b3a9d74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:11.822892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b41c6-ab1b-4064-af9f-171b3a9d74af None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:11.823216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b41c6-ab1b-4064-af9f-171b3a9d74af None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:11.823519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b41c6-ab1b-4064-af9f-171b3a9d74af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:11.824189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b41c6-ab1b-4064-af9f-171b3a9d74af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:11.825622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b41c6-ab1b-4064-af9f-171b3a9d74af None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:11.864992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 357/719] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:30:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:12.317688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a263277-1ced-4c2c-8a3c-3e50e745fccb None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:12.318393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a263277-1ced-4c2c-8a3c-3e50e745fccb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:12.318905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a263277-1ced-4c2c-8a3c-3e50e745fccb None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:12.320772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 363/720] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:12.326709 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:12.327655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:12.328135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:12.347571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:12.349105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:12.431935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 358/721] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:12.707932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4033d-92e4-4012-a8a0-a7323c11c3f6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:12.707932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4033d-92e4-4012-a8a0-a7323c11c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:12.707932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4033d-92e4-4012-a8a0-a7323c11c3f6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:12.707932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 364/722] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:12.709025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:12.714464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:12.714776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:12.715014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:12.737628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:12.739339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:12.758121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:12.827904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b5f86-c2dc-4fc8-8369-96f5ba707831 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:12.839440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 365/723] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:12.870472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:12.871098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:12.871369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:12.871672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:12.872272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:12.873598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:12.882187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:12.883119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:12.883979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 359/724] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:12.892793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:13.024724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332616a3-c0a1-4c3c-a4a5-5fc8b05d5238 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:13.045170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:13.081411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:13.082397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:13.082876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:13.083343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:13.085629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:13.086203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:13.087907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:13.105399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 366/725] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:13.115927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:13.159862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:13.328181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b2cd7-426b-4fd1-8984-e78c194265dd None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:13.331947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:13.364677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffefa67-dc8a-43eb-9a69-3151f2cd0cc4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:13.402095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:13.403350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:13.407119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:13.409889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:13.410787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:13.412264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:13.413853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:13.427215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 360/726] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:13.451608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:13.452670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:13.453172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:13.453654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] PATH_INFO: `/v3/services/image` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:13.455011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:13.460268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:13.483804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:13.484937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:13.485786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:13.489437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 367/727] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Oct 11 06:30:13 2022] GET /identity/v3/services/image => generated 86 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:13.497301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:13.669425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af93eb-7459-4487-a541-f6f94e8b9579 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:13.706796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:13.707801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:13.708298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:13.708803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:13.709625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:13.710220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:13.712552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a10a-cebf-4b11-be32-94328ad263d3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:13.718079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 361/728] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:13.723939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:13.774703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:13.833961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98a5cd-d945-4394-bfd9-c65d3bf46b6b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:13.868804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:13.869786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:13.870361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:13.870851 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:13.871719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] RBAC: Adding query filter params (type=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:13.872219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:13.873794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd2317-3d9e-4bca-89e3-d1859860d9cf None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:13.883537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 368/729] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:14.064199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284deb22-1d4d-4ca1-9a2d-e4adf52540be None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:14.064507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284deb22-1d4d-4ca1-9a2d-e4adf52540be None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:14.064749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284deb22-1d4d-4ca1-9a2d-e4adf52540be None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:14.066363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 362/730] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:14.071877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c084-e78c-4365-8adb-74a5ed9dd827 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:14.072161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c084-e78c-4365-8adb-74a5ed9dd827 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:14.072436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c084-e78c-4365-8adb-74a5ed9dd827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:14.091668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1d98c084-e78c-4365-8adb-74a5ed9dd827 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:14.093143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1d98c084-e78c-4365-8adb-74a5ed9dd827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:14.226248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 369/731] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:14.544159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284deb22-1d4d-4ca1-9a2d-e4adf52540be None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:14.580674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881324-339b-4dc6-b519-3fec04ca9d44 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:14.581283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881324-339b-4dc6-b519-3fec04ca9d44 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:14.583195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8881324-339b-4dc6-b519-3fec04ca9d44 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:14.593170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 370/732] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:14.594942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:14.596484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:14.598910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:14.623775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:14.635742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:14.640484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:14.678635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:14.721363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284deb22-1d4d-4ca1-9a2d-e4adf52540be None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:14.732035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 371/733] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:14.780396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:14.781008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:14.781286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:14.781556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:14.782296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:14.784019 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:14.786945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING keystone.server.flask.common [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 11 06:30:14.804909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.sql.core [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 06:30:14.804909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 06:30:14.804909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [parameters: {'id': '369bd8fafb2c4f21b4ec62659bb82a54', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 11 06:30:14.804909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81484) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 369bd8fafb2c4f21b4ec62659bb82a54.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 369bd8fafb2c4f21b4ec62659bb82a54. Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:30:14.812650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result.read() Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 06:30:14.813968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application yield resource Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise self.value Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application session.commit() Oct 11 06:30:14.814916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.session.flush() Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application compat.raise_( Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 06:30:14.816736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 06:30:14.818863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application result.read() Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:30:14.822558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [parameters: {'id': '369bd8fafb2c4f21b4ec62659bb82a54', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:14.824889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:14.826726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 369bd8fafb2c4f21b4ec62659bb82a54. Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:14.828808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 363/734] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Oct 11 06:30:14 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:14.830603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:14.937970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f9c4-527a-4b1b-8127-3d752d3c7c13 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:14.977307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:14.978337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:14.978855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:14.979383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:14.986962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:14.988685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:30:15.000233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:15.001239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:15.002098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:15.004409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 372/735] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:15.010443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:15.025174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:15.056503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:15.116503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:15.136725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7140b-482f-456e-924f-0ecc7e3bf01f None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:15.142036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b947055-c448-4dc9-9ac2-be0a78673afd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:15.181469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:15.182118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.182396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.182672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.183336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:15.183684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:15.185344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.186704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:15.187713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.188238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.189272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.190479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:15.192065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:15.202452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:15.203404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:15.204593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:15.207019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 373/736] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:15.207559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 364/737] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:15.216803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:15.328049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:15.338003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee0ea8-39a1-404c-8d64-9b6b12e3b707 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:15.385116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:15.386031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.386459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.386881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.387746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:15.388181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:15.389724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.399541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 374/738] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:15.401876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:15.524832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3faf-9b4e-41c4-83de-5d83aadf9294 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:15.563320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:15.565267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.565801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.566292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.567164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:15.567701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:15.569269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.575003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 365/739] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:15.581170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:15.691447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802c1b9-fe81-4cbc-a272-1539943043d5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:15.727574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:15.728190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.728470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.728761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.729456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] RBAC: Adding query filter params (interface=admin, service_id=1dfd9690103e47f29e7d6868e5f64038, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:15.729767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:15.731138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167d0cb-8c98-493f-847d-c1773c82e6b1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.741729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 375/740] 10.222.0.39 () {56 vars in 1249 bytes} [Tue Oct 11 06:30:15 2022] GET /identity/v3/endpoints?service_id=1dfd9690103e47f29e7d6868e5f64038&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:15.744505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:15.763215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:15.861324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dcbe7-1b8c-4c6a-b858-fb3a0795fddf None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:15.896005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4c16-d28d-4d83-8861-db78028a3d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:15.896620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4c16-d28d-4d83-8861-db78028a3d45 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.896902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4c16-d28d-4d83-8861-db78028a3d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.897179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4c16-d28d-4d83-8861-db78028a3d45 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.897791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea4c16-d28d-4d83-8861-db78028a3d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:15.899649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea4c16-d28d-4d83-8861-db78028a3d45 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:15.911569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 366/741] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:30:15 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:30:15.917877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50959b59-8066-4bde-b96a-1ad15056237f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.918161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50959b59-8066-4bde-b96a-1ad15056237f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.918430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50959b59-8066-4bde-b96a-1ad15056237f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.919826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 376/742] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:15.925933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e95623-83f0-4f16-a9e0-6bbbef05e799 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:15.926219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e95623-83f0-4f16-a9e0-6bbbef05e799 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:15.926492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e95623-83f0-4f16-a9e0-6bbbef05e799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:15.952557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b7e95623-83f0-4f16-a9e0-6bbbef05e799 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:15.954205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b7e95623-83f0-4f16-a9e0-6bbbef05e799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:16.028867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 367/743] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:16.288962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3292-b572-4bf5-a205-12791b4458d5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:16.289170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3292-b572-4bf5-a205-12791b4458d5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:16.289446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3292-b572-4bf5-a205-12791b4458d5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:16.291104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 377/744] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:16.295957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:16.296240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:16.296509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:16.304245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:16.316014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:16.317364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3292-b572-4bf5-a205-12791b4458d5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:16.318889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:16.343556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:16.411572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 368/745] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:16.444476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3292-b572-4bf5-a205-12791b4458d5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:16.480019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17fd20-ea8c-4c4f-8de8-27d395567e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:16.480965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17fd20-ea8c-4c4f-8de8-27d395567e68 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:16.481458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17fd20-ea8c-4c4f-8de8-27d395567e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:16.481997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17fd20-ea8c-4c4f-8de8-27d395567e68 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:16.482879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17fd20-ea8c-4c4f-8de8-27d395567e68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:16.484505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17fd20-ea8c-4c4f-8de8-27d395567e68 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:16.507608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 378/746] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:16 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:16.681976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:16.790806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf828f8a-a317-4b2f-acc9-e237d8e4efdd None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:16.826206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:16.835119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf49912-c3f3-4c30-8cc8-27385967c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:16.835806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf49912-c3f3-4c30-8cc8-27385967c56e None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:16.836081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf49912-c3f3-4c30-8cc8-27385967c56e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:16.836345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf49912-c3f3-4c30-8cc8-27385967c56e None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:16.837023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf49912-c3f3-4c30-8cc8-27385967c56e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:16.838414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf49912-c3f3-4c30-8cc8-27385967c56e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:16.855606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 369/747] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:16 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:17.577472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca157f-5a81-4591-8a65-51379aa71128 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:17.578029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca157f-5a81-4591-8a65-51379aa71128 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:17.578029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca157f-5a81-4591-8a65-51379aa71128 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:17.580220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 379/748] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:17.585430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:17.585720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:17.585992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:17.604412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:17.605603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:17.686185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 370/749] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:17.960745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca157f-5a81-4591-8a65-51379aa71128 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:18.059861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:18.074579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca157f-5a81-4591-8a65-51379aa71128 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:18.111633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:18.112547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.113322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.113742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.114542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:18.116175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:18.118852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING keystone.server.flask.common [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 11 06:30:18.132809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.sql.core [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 06:30:18.132809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 11 06:30:18.132809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [parameters: {'user_id': 'cd2a4fc24d2842faa00b2c1842ab4abe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 11 06:30:18.132809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81485) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-37df2636-e9f2-4680-8418-37dc13d20b6e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Oct 11 06:30:18.145546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:30:18.146568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application yield resource Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise self.value Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application session.commit() Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 06:30:18.147778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.flush() Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application compat.raise_( Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 06:30:18.148664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:30:18.149596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:30:18.150463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cd2a4fc24d2842faa00b2c1842ab4abe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:18.151266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 11 06:30:18.152078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 11 06:30:18.152977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 11 06:30:18.152977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 11 06:30:18.152977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 06:30:18.152977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 06:30:18.152977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 11 06:30:18.152977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:18.155072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 380/750] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Oct 11 06:30:17 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:18.161606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:18.198963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b49d6a-0e95-4c8a-bf37-5359167f73de None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.199535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b49d6a-0e95-4c8a-bf37-5359167f73de None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.199975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b49d6a-0e95-4c8a-bf37-5359167f73de None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.210514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 381/751] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:18.212539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.213083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.213602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.232809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:18.234668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:18.268817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198258e-77d6-40b5-ac2f-4bb959b74bfb None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:18.295790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:18.307018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:18.307823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.308208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.308582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.322419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:18.322419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:18.322778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 382/752] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:30:18.331018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:18.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 06:30:18.333120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:18.338882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 371/753] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:30:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:18.342589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:18.425124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:18.456753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf8200-3a25-48d3-9b6e-39c26dc1ae89 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:18.493170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:18.499456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.499456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.499456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.499456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:18.499456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:18.499456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:18.533122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 383/754] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Oct 11 06:30:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:18.605551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:18.738600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f8a8b-8cd6-4950-9fee-cfe700d3bf22 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:18.773605 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:18.774168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.774445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.774713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.775303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:18.776690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-398752f9-90a1-4004-854c-639b8a4dceeb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:18.786142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:18.787184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:18.788238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 372/755] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Oct 11 06:30:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:18.795399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:18.800917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8221f-3d68-4aa6-a6d4-f228ba783a84 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.800917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8221f-3d68-4aa6-a6d4-f228ba783a84 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.801169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8221f-3d68-4aa6-a6d4-f228ba783a84 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.802691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 373/756] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:18.807771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.808054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.808321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.833057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:18.834260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:18.875595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:18.910584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 374/757] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:18.922398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f42cd6-76cc-42b0-b82f-91119420b7d7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:18.955016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:18.961774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:18.962384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:18.962661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:18.962965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:18.963581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:18.963883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:18.965245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8c1d9-2521-4f11-9244-bd156ed20e06 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:18.975508 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 384/758] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:18.977054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.084732 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf167c-4d06-4c8f-885c-2d8e9cd05702 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.109343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e3324-54b1-46b1-a560-f93dae2771b1 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.109626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e3324-54b1-46b1-a560-f93dae2771b1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.109903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e3324-54b1-46b1-a560-f93dae2771b1 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.112726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 385/759] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:30:19.116715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.117002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.117280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.121617 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.122431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.122901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.123337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.124176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:19.124477 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.125873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.135189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 375/760] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Oct 11 06:30:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:19.135337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:19.136388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:19.136612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.176503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:19.207803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 386/761] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:19 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:19.211739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.252461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e53-ee2d-4766-b226-8c64d4f0e553 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.287788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.288403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.288683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.288958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.289627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.290984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.308451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 376/762] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:30:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:30:19.317116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26015620-be64-46e0-a2dd-5401b8ecafbb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.346555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:19.370662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.371388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.371730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.372032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.372683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.374159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:19.383975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:19.385042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:19.385863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:19.387752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 387/763] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:19.391203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.487777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.505631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:19.515631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ce458-906d-426d-8a98-ad921c4e9319 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.552413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.553020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.553303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.553576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.554217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:19.554516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.555932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.566235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 377/764] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:19.567749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.606484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9ada5-5fd8-4d40-9af2-b10b226b648f None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.643558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.644197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.644476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.644750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.645331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.646697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:19.655675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:19.656644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:19.658274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 388/765] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:19.664750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.693197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3713f2e-59cb-4a1e-9fdf-8316aac505b7 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.719636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:19.730247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.730856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.731133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.731398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.732043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:19.732342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.733695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.738778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 378/766] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:19.744232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.771382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:19.779102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff86ed8-a9e1-48c0-821f-fd26ec3820d8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.816971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.817623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.817839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.818107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.818730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] RBAC: Adding query filter params (name=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:19.819029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.820519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.825635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 389/767] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:19.831807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.861678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d183ef-3162-430a-bcea-2e76ef30a322 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.900786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.901398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.901674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.901939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.902636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] RBAC: Adding query filter params (interface=public, service_id=984cb4159b574b31b12622ec74c7c11d, region_id=RegionOne) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:19.902938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.904386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.910753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 379/768] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/endpoints?service_id=984cb4159b574b31b12622ec74c7c11d&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:19.916254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:19.944494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1c224-e654-4556-ab93-212200b38881 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:19.961066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:19.984518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:19.985153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:19.985426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:19.985692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:19.986310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] RBAC: Adding query filter params (type=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:19.986601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:19.987991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:19.993217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 390/769] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:19.998848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:20.010224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:20.028626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59d49-1416-4ad0-a74d-d7b424767fa0 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:20.064974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:20.065725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:20.066106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:20.066449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:20.067111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:20.068596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:20.075297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 380/770] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:30:20.104096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844e852-9588-48dd-865e-c8cdddbd0a24 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:20.146027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:20.146707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:20.147081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:20.147474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:20.148297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] RBAC: Adding query filter params (interface=public, service_id=5f61a2211c9740c5a2e846573e439944, region_id=RegionOne) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:20.148709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:20.150202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba17252-2b31-4756-b5ed-69832954765a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:20.161189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 391/771] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:30:19 2022] GET /identity/v3/endpoints?service_id=5f61a2211c9740c5a2e846573e439944&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:20.162434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:20.277049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e9f98-bfc9-48c9-95e2-27ad4ea63358 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:20.309933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928e679-bd5d-4d89-bf9f-342cbcd1c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:20.310542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928e679-bd5d-4d89-bf9f-342cbcd1c757 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:20.310820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928e679-bd5d-4d89-bf9f-342cbcd1c757 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:20.311092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928e679-bd5d-4d89-bf9f-342cbcd1c757 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:20.311725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928e679-bd5d-4d89-bf9f-342cbcd1c757 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:20.313082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928e679-bd5d-4d89-bf9f-342cbcd1c757 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:20.320321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 381/772] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:30:20 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:30:21.494048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7fa32-8cfe-4149-a689-9a55b0697544 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:21.494578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7fa32-8cfe-4149-a689-9a55b0697544 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:21.494578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7fa32-8cfe-4149-a689-9a55b0697544 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:21.496076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 392/773] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:21.501488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:21.501904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:21.502043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:21.519983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:21.521258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:21.530438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:21.592737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 382/774] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:21.898319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7fa32-8cfe-4149-a689-9a55b0697544 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:21.946729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:22.020279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7fa32-8cfe-4149-a689-9a55b0697544 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:22.060969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2b57a-231b-421c-a326-f94a2636e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:22.061606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2b57a-231b-421c-a326-f94a2636e8ef None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.061885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2b57a-231b-421c-a326-f94a2636e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.062157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2b57a-231b-421c-a326-f94a2636e8ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.063818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2b57a-231b-421c-a326-f94a2636e8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:22.065367 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2b57a-231b-421c-a326-f94a2636e8ef None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:22.067457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 393/775] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:22.072929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:22.185899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d5cd-0260-427f-b668-fb2e312e4779 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:22.220322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a42d16-666a-418b-b26f-5fb9ee63c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:22.220936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a42d16-666a-418b-b26f-5fb9ee63c943 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.221216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a42d16-666a-418b-b26f-5fb9ee63c943 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.221488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a42d16-666a-418b-b26f-5fb9ee63c943 None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.222156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a42d16-666a-418b-b26f-5fb9ee63c943 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:22.223605 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a42d16-666a-418b-b26f-5fb9ee63c943 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:22.240171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:22.255564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 383/776] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:22 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:22.264863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4f9-68a9-485a-ad83-02291d2625d8 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.265165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4f9-68a9-485a-ad83-02291d2625d8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.265552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4f9-68a9-485a-ad83-02291d2625d8 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.267922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 394/777] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:22.273742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5398f2-d48d-4d6c-8b05-09a5d4a50cb9 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.274041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5398f2-d48d-4d6c-8b05-09a5d4a50cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.274307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5398f2-d48d-4d6c-8b05-09a5d4a50cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.293729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-bb5398f2-d48d-4d6c-8b05-09a5d4a50cb9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:22.294952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bb5398f2-d48d-4d6c-8b05-09a5d4a50cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:22.376926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 384/778] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:22 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:22.510246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefcb39-69eb-44c0-b617-dd8921933c4e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.510742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefcb39-69eb-44c0-b617-dd8921933c4e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.510742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefcb39-69eb-44c0-b617-dd8921933c4e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.512242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 395/779] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:22.517321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54982614-a6fe-4185-ac13-82d437476129 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.517702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54982614-a6fe-4185-ac13-82d437476129 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.517874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54982614-a6fe-4185-ac13-82d437476129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.536894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-54982614-a6fe-4185-ac13-82d437476129 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:22.538075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-54982614-a6fe-4185-ac13-82d437476129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:22.609457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 385/780] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:22 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:22.645764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefcb39-69eb-44c0-b617-dd8921933c4e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:22.751835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefcb39-69eb-44c0-b617-dd8921933c4e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:22.761407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:22.793356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:22.794101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.794446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.794787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.795467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:22.796931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:22.806574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:22.807552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:22.812462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 396/781] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:22.813789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54982614-a6fe-4185-ac13-82d437476129 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:22.871061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:22.877665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:22.925353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54982614-a6fe-4185-ac13-82d437476129 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:22.958932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:22.959566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:22.959844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:22.960112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:22.960764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:22.961063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:22.962438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:22.967753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 386/782] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:22.973603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:22.994274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4237f-1e04-4cf8-ac40-cdf184a1b2e9 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:23.030793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:23.031356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:23.031680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:23.032728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] PATH_INFO: `/v3/services/image` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:23.033780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:23.035181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:23.045160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:23.046149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:23.046978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 397/783] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Oct 11 06:30:22 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:23.052583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:23.089861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd968fd-3409-4bb3-b8cf-dcc5d3dc3d9b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:23.092097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:23.114109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:23.129015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:23.129621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:23.129900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:23.130169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:23.130794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:23.131093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:23.132507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:23.137716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 387/784] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Oct 11 06:30:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:23.144171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:23.170112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d948ae-dcd6-46f1-9830-14df7b4d0982 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:23.212543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:23.213095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:23.213377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:23.213646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:23.214273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:23.214571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:23.215979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:23.221491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 398/785] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:23.226863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:23.255868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597b64c-0b10-4c3f-9851-9172fdb623b2 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:23.293306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:23.294119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:23.294550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:23.294971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:23.295821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:23.297643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:23.322948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 388/786] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:30:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:23.336928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:23.341192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1f8cf-fda5-4a92-95d2-c7d5eca9e5a2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:23.387286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:23.387943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:23.388223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:23.388493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:23.389156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] RBAC: Adding query filter params (type=image) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:23.389459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:23.390854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c95-3cfa-4c04-8a2f-e1fffa4ef97a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:23.403920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 399/787] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:23.404133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:23.459711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:23.525947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146ab0e-f74d-46d5-bc16-290f88a0f84a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:23.561612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef5bb-2b57-4c34-8b83-a2f6b8e71940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:23.562564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef5bb-2b57-4c34-8b83-a2f6b8e71940 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:23.563558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef5bb-2b57-4c34-8b83-a2f6b8e71940 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:23.563845 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef5bb-2b57-4c34-8b83-a2f6b8e71940 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:23.564532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef5bb-2b57-4c34-8b83-a2f6b8e71940 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:23.565886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef5bb-2b57-4c34-8b83-a2f6b8e71940 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:23.586806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 389/788] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:30:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:24.477018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336f2b5-c6c5-435a-8710-e91cc0c42acc None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:24.477549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336f2b5-c6c5-435a-8710-e91cc0c42acc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:24.477549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336f2b5-c6c5-435a-8710-e91cc0c42acc None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:24.479140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 400/789] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:24.484968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:24.485764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:24.486031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:24.504627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:24.505824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:24.577539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 390/790] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:24.852467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336f2b5-c6c5-435a-8710-e91cc0c42acc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:24.955933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336f2b5-c6c5-435a-8710-e91cc0c42acc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:24.989105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a797-5785-4b18-8a0d-3720b93f5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:24.989736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a797-5785-4b18-8a0d-3720b93f5051 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:24.990018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a797-5785-4b18-8a0d-3720b93f5051 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:24.990291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a797-5785-4b18-8a0d-3720b93f5051 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:24.991398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a797-5785-4b18-8a0d-3720b93f5051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:24.992863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a797-5785-4b18-8a0d-3720b93f5051 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:24.994585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 401/791] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:25.000122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:25.027538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:25.105276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ac557-2ab7-4988-8ab0-561e5a17b88c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:25.142817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2af6c-36c4-44ed-a691-eaade0c1affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:25.143452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2af6c-36c4-44ed-a691-eaade0c1affd None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:25.143734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2af6c-36c4-44ed-a691-eaade0c1affd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:25.144002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2af6c-36c4-44ed-a691-eaade0c1affd None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:25.144735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2af6c-36c4-44ed-a691-eaade0c1affd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:25.146155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2af6c-36c4-44ed-a691-eaade0c1affd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:25.168118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 391/792] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:30:24 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:25.578465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921064f-a2d9-4c65-b4e9-dfa6d1864b65 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:25.579039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921064f-a2d9-4c65-b4e9-dfa6d1864b65 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:25.579039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921064f-a2d9-4c65-b4e9-dfa6d1864b65 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:25.580416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 402/793] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:25.585654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19948b10-0764-413f-8df2-2cb56859c536 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:25.586067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19948b10-0764-413f-8df2-2cb56859c536 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:25.586236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19948b10-0764-413f-8df2-2cb56859c536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:25.604095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-19948b10-0764-413f-8df2-2cb56859c536 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:25.605281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-19948b10-0764-413f-8df2-2cb56859c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:25.691632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 392/794] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:25.762551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8123b0-466f-41c4-a0ee-25ffbbf8ba19 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:25.762872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8123b0-466f-41c4-a0ee-25ffbbf8ba19 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:25.763106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8123b0-466f-41c4-a0ee-25ffbbf8ba19 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:25.768988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 403/795] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:30:25.770881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:25.771174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:25.771471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:25.790376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:25.791586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:25.793612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:25.861357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 393/796] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:25.950667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8123b0-466f-41c4-a0ee-25ffbbf8ba19 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:25.981123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.054303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8123b0-466f-41c4-a0ee-25ffbbf8ba19 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.091671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.092424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.092797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.093144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.105336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:26.110483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:26.111458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:26.112350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.113851 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 404/797] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:30:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:26.118673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.132086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.225320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.230618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5532d-0ea1-4c98-9bc9-857998cc4eae None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.239241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933429aa-aa99-436d-9bd0-fa14bd033815 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.263133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.263766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.264050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.264329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.265006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:26.265307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.267199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.272113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.275501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 394/798] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:26.279601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.279601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.279601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.279601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.279929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.281000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.281354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.289610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.sql.core [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 06:30:26.289610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 06:30:26.289610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [parameters: {'id': 'c84b8da1fdc247a2a8d21a4fe5ddcbaf', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 06:30:26.289610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81485) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:30:26.297228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 06:30:26.298153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application yield resource Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application next(self.gen) Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise self.value Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application session.commit() Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 06:30:26.299111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.session.flush() Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application compat.raise_( Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 06:30:26.300505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 06:30:26.301374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application conn.query(q) Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application result.read() Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 06:30:26.302341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [parameters: {'id': 'c84b8da1fdc247a2a8d21a4fe5ddcbaf', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 06:30:26.303919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:26.305568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 405/799] 10.222.0.39 () {60 vars in 1137 bytes} [Tue Oct 11 06:30:26 2022] POST /identity/v3/domains => generated 190 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:26.307135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.400012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14c86b-fcc2-48b1-ab83-91035aa89770 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.414981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e718cf4-4431-447e-b871-b6052ca5fb1b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.436236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.436862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.437136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.437401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.448712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.450134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.459808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.460716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.461156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.461584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:26.464928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:26.466033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:26.467071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:26.467071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:26.467071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:26.467071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 395/800] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:26.471762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.476519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.478184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:26.497447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:26.498541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:26.499394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:26.499394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:26.499394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:26.505088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 406/801] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:26.505759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.509464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.529400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.577663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47221da8-0e51-42e3-8e2a-be73b72d4586 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.614285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.614985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.615263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.615566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.616207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:26.616506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.617861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.618247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93841fb-3287-4b8f-b0ed-8dcc68817d0b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.631950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 396/802] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:26.637465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.659828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.661277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.661785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.662528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.663671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:26.664603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.666725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.683027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 407/803] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:26.749447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa634b-4b01-4e57-81cb-d42c0ee13c29 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.764530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.788419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.789062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.789335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.789598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.797382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.798845 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:30:26.811304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:26.812592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:26.814024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 397/804] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:26.823443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:26.888319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:26.933942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9e28e-ff06-4e60-b52d-68a9099d41fd None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:26.972863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:26.973476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:26.973753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:26.974820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:26.974820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:26.975102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:26.976589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:26.995748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 408/805] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Oct 11 06:30:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:27.001692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:27.101720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8373d-756c-4c36-b3a6-4de28b033642 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:27.135043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:27.135697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.135980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.136248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.146211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:27.147702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:27.161523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:27.162607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:27.163525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:27.163525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:27.163525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:27.167218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 398/806] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:27.168516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:27.269197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b62746-ecd4-4ce6-9676-fab64af3959b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:27.293468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:27.305673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:27.306275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.306550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.306818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.307479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:27.307781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:27.309147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd6dd-9a7c-4412-8346-4a8cd60eb6f3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:27.319927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 409/807] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:27.325642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:27.351097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:27.373600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501225-2af9-43ee-9372-01480fd16925 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.373600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501225-2af9-43ee-9372-01480fd16925 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.373600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501225-2af9-43ee-9372-01480fd16925 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.374134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 410/808] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:27.379030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.379317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.379621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.397152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:27.398334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:27.428533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b17b2f-d689-4591-8a51-7e6ef6fc2a43 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:27.460835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:27.461460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.461740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.462010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.465763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 411/809] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:27.468465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:27.470407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:30:27.481515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:27.482584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:27.483869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 399/810] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:27.488932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:27.587611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd721-ba1c-40ec-b8d9-e971693751ec None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:27.600950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:27.621061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:27.621653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.621920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.622186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.622866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:27.623163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:27.624608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:27.635532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 412/811] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:27.642190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:27.726922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:27.733945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:27.743088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320837e-51d4-4f0d-abd2-041eaa896b65 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:27.775854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:27.776498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.776772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.777044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/fbf7156a9d9448a6987fe7999543cedb/roles/be335f32244b4b4991452bafc82027f5` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.779157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=fbf7156a9d9448a6987fe7999543cedb, role_id=be335f32244b4b4991452bafc82027f5)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:27.780633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:27.794466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 400/812] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:30:27 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/fbf7156a9d9448a6987fe7999543cedb/roles/be335f32244b4b4991452bafc82027f5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:30:27.838862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906a59-43df-4302-b854-f3d61a18e91a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:27.875960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:27.876598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:27.876871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:27.877135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:27.889038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:27.894402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:27.895503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:27.896331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 413/813] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:30:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:27.901892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:28.005671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb360-0959-40f9-b503-8ae52e925b57 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:28.038851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:28.039478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:28.039755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:28.040047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:28.040718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:28.041014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:28.042357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:28.050643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 401/814] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:30:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:28.056416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:28.087682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:28.159480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e535f-dda1-4044-b556-f60a5e2fe0ec None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:28.192292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:28.192919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:28.193196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:28.193467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] PATH_INFO: `/v3/users/c1702438641f43b28bd545eeea597644` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:28.206931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] RBAC: Authorizing `identity:get_user(user_id=c1702438641f43b28bd545eeea597644)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:28.208398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:28.210009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 414/815] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:28 2022] GET /identity/v3/users/c1702438641f43b28bd545eeea597644 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:28.215776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:28.226640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:28.315843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9570830-84c5-4b62-8775-6a1e8294b76d None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:28.350554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e835-2068-4862-b712-8047d4ceaa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:28.351181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e835-2068-4862-b712-8047d4ceaa07 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:28.351493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e835-2068-4862-b712-8047d4ceaa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:28.351798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e835-2068-4862-b712-8047d4ceaa07 None admin] PATH_INFO: `/v3/projects/a122f8edfaf149248e7a575a66409992` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:28.352902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e835-2068-4862-b712-8047d4ceaa07 None admin] RBAC: Authorizing `identity:get_project(project_id=a122f8edfaf149248e7a575a66409992)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:28.354315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e835-2068-4862-b712-8047d4ceaa07 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:28.355971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 402/816] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:30:28 2022] GET /identity/v3/projects/a122f8edfaf149248e7a575a66409992 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:28.362307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:28.454490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:28.464659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88615d-6c9e-411e-98b8-1ce6ca93d691 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:28.501372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6813304-4d21-46c5-a801-a146e75449d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:28.502020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813304-4d21-46c5-a801-a146e75449d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:28.502294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813304-4d21-46c5-a801-a146e75449d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:28.502558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813304-4d21-46c5-a801-a146e75449d8 None admin] PATH_INFO: `/v3/projects/a122f8edfaf149248e7a575a66409992/users/c1702438641f43b28bd545eeea597644/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:28.504451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813304-4d21-46c5-a801-a146e75449d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a122f8edfaf149248e7a575a66409992, user_id=c1702438641f43b28bd545eeea597644, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:28.505899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813304-4d21-46c5-a801-a146e75449d8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:28.518099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 415/817] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:30:28 2022] PUT /identity/v3/projects/a122f8edfaf149248e7a575a66409992/users/c1702438641f43b28bd545eeea597644/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:30:29.939504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc214-52cd-4461-b3a6-aeb68b650fa8 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:29.940027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc214-52cd-4461-b3a6-aeb68b650fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:29.940027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc214-52cd-4461-b3a6-aeb68b650fa8 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:29.941547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 403/818] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:29.946868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:29.947283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:29.947520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:29.966000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:29.967179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:30.036520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 416/819] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:30.309572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc214-52cd-4461-b3a6-aeb68b650fa8 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:30.387952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:30.419317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc214-52cd-4461-b3a6-aeb68b650fa8 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:30.454568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:30.455200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:30.455506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:30.455780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:30.466434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:30.471918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:30.472823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:30.473822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 404/820] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Oct 11 06:30:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:30.479109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:30.572818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:30.593670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9e4bc-845a-48cf-8114-32452c01c026 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:30.629828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:30.630439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:30.630714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:30.630978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:30.631673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:30.631968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:30.633324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164effef-42fe-44c3-99e3-fd37c9eda3d9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:30.642391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 417/821] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Oct 11 06:30:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:30.648498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:30.658852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0c868-5bda-48f7-9d4b-9255d8556c31 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:30.659134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0c868-5bda-48f7-9d4b-9255d8556c31 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:30.659403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0c868-5bda-48f7-9d4b-9255d8556c31 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:30.660809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 418/822] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:30:30.666019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:30.666294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:30.666564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:30.684431 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:30.685821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:30.761742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611e3db-06f0-430a-adfa-0be6bfb0213b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:30.763587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 419/823] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:30.807170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:30.807850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:30.808131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:30.808405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:30.811009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:30.820307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:30.821742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:30.837027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:30.837951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:30.838795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:30.838795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:30.838795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:30.838795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 405/824] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:30.843993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:30.920192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:30.951129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419c76f-8dc1-44bd-979f-cd02246c2358 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:30.985911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:30.986881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:30.987383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:30.987884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:30.988739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:30.989239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:30.990813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.003369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 420/825] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:31.008873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.035164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.148407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdcca6-7bab-4079-8c28-62ca1b95815d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.182561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.188133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.188791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.189081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.189571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.202018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:31.206970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:31.208011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:31.208916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 421/826] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:31.214279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.246648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c53eeb-0bf2-4b43-9eda-669d220a5040 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.285873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.286512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.286796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.287053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.294636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:31.296290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:30:31.308801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:31.309743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:31.310631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 406/827] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:31.315719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.330187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb2ade-fa34-480c-995c-24c498fe5324 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.368813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.369524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.369976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.370252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.370920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:31.371218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:31.372654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.373644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.383309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 422/828] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:31.389208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.441040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf3d19-f151-429c-969c-c31b360d6640 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.458989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.478380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.478987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.479260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.479593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.480352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:31.480649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:31.482518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.500676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 407/829] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:31.506438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.509924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea3ece-6055-426c-84b7-d589163cef8b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.550289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.550289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.550289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.550289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] PATH_INFO: `/v3/users/c1702438641f43b28bd545eeea597644` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.551272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] RBAC: Authorizing `identity:get_user(user_id=c1702438641f43b28bd545eeea597644)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:31.552958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.559216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 423/830] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/users/c1702438641f43b28bd545eeea597644 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:31.560246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.621806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019102d0-47ae-4f1a-b866-d3ef2259ffd3 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.625368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.665158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.665824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.666303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.666590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.669678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8aea6-61df-45f0-bf03-87375a3dee89 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:31.695083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.708410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:31.709182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:31.710228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:31.710513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:31.746063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:31.747565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.787528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:31.788981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:31.854558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 408/831] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 11 06:30:31.860371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.863068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.895912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 424/832] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 11 06:30:31.901633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:31.930131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:31.974261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df85230-5275-4a3d-b5f6-93c654d24f67 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.006145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c00877-5a7b-4e9b-85a6-2a0841f1d8e6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.009718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.010430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.010781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.011121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.021960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.023517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:30:32.039336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:32.040520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:30:32.041409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:30:32.041409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:30:32.041409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:32.041409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 409/833] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:32.044233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.045063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.045343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.045621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] PATH_INFO: `/v3/projects/a122f8edfaf149248e7a575a66409992` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.046479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:32.046753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] RBAC: Authorizing `identity:get_project(project_id=a122f8edfaf149248e7a575a66409992)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.048293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.054832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 425/834] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:30:31 2022] GET /identity/v3/projects/a122f8edfaf149248e7a575a66409992 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:32.055855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:32.111208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:32.157705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:32.159734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511f93a-f70f-4f42-95f7-eb6a0275a3a5 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.169926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332508d-f4f4-473d-a43d-0e2f0ca019fb None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.195292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.196117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.196478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.196733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.197391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:32.197890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.199352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.211586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 410/835] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:32.211764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.212058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.212332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.212602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.213337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=a122f8edfaf149248e7a575a66409992, user.id=c1702438641f43b28bd545eeea597644) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:32.213639 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.215235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.217273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:32.222361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 426/836] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:30:32 2022] GET /identity/v3/role_assignments?user.id=c1702438641f43b28bd545eeea597644&scope.project.id=a122f8edfaf149248e7a575a66409992&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:32.327902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58170c-364e-4863-91d7-86bab8da569b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.346789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:32.369142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.369794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.370072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.370342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.378053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.379609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:30:32.392979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:32.393968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:32.395067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 411/837] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:30:32.400302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:32.514791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e09d3-9dab-4049-be93-0a5ec8e4e84d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.551904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.552513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.552792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.553061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.553751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:32.554050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.555501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdef83-d8da-4354-bb55-e6ba72232430 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.568979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 427/838] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:32.574697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:32.684144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fcbb9-6c3e-4bed-b1a4-54b6c536e452 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:32.718924 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:32.719828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:32.720261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:32.720693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:32.721419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] RBAC: Adding query filter params (role.id=be335f32244b4b4991452bafc82027f5, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=fbf7156a9d9448a6987fe7999543cedb) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:32.721725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:32.723148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37722861-7c64-4e6a-87d2-ea298dac16a5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:32.730416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 412/839] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:30:32 2022] GET /identity/v3/role_assignments?user.id=fbf7156a9d9448a6987fe7999543cedb&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=be335f32244b4b4991452bafc82027f5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:30:34.425038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe0025-00a7-4095-b009-896e374375e4 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:34.425038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe0025-00a7-4095-b009-896e374375e4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:34.425038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe0025-00a7-4095-b009-896e374375e4 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:34.426690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 428/840] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:34.432402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:34.432835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:34.433001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:34.451526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:34.452780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:34.487817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:34.522396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 413/841] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:34.792140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe0025-00a7-4095-b009-896e374375e4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:34.808691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:34.897695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe0025-00a7-4095-b009-896e374375e4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:34.932536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a9649-02e2-4d0e-8913-15545ca4a2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:34.933165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a9649-02e2-4d0e-8913-15545ca4a2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:34.933444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a9649-02e2-4d0e-8913-15545ca4a2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:34.933711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a9649-02e2-4d0e-8913-15545ca4a2b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:34.934835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a9649-02e2-4d0e-8913-15545ca4a2b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:34.936304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a9649-02e2-4d0e-8913-15545ca4a2b3 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:34.937876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 429/842] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:34.943810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:35.040697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd782-0545-4aa5-941d-82e36c797e95 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:35.073976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74819e8c-10d1-4ac1-a9a4-b305b84ba056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:35.074514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74819e8c-10d1-4ac1-a9a4-b305b84ba056 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:35.074787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74819e8c-10d1-4ac1-a9a4-b305b84ba056 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:35.075054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74819e8c-10d1-4ac1-a9a4-b305b84ba056 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:35.075752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74819e8c-10d1-4ac1-a9a4-b305b84ba056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:35.077159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74819e8c-10d1-4ac1-a9a4-b305b84ba056 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:35.099096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 414/843] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:30:34 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:37.253160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43ab4f-ae9c-4a1f-a51c-e4380c1b9e40 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:37.253689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43ab4f-ae9c-4a1f-a51c-e4380c1b9e40 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:37.253689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43ab4f-ae9c-4a1f-a51c-e4380c1b9e40 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:37.255052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 430/844] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:37.260074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:37.260469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:37.260615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:37.277993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:37.279166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:37.299306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:37.345234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 415/845] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:37.614864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43ab4f-ae9c-4a1f-a51c-e4380c1b9e40 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:37.694764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:37.713536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43ab4f-ae9c-4a1f-a51c-e4380c1b9e40 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:37.746803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:37.747437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:37.747717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:37.747987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] PATH_INFO: `/v3/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:37.749354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] RBAC: Authorizing `identity:get_role(role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:37.750769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:37.752106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 431/846] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:37 2022] GET /identity/v3/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:37.757790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:37.850418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03e22f-7bb0-4ee8-ac1e-548a36a340ee None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:37.885183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:37.885750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:37.886029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:37.886299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] PATH_INFO: `/v3/users/d35f1c163eb344658e2c75d1a876fa01` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:37.899357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] RBAC: Authorizing `identity:get_user(user_id=d35f1c163eb344658e2c75d1a876fa01)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:37.900826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:37.902425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 416/847] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:37 2022] GET /identity/v3/users/d35f1c163eb344658e2c75d1a876fa01 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:37.909126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:38.009083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27424c-f9b2-4373-b357-103e7f97f3e8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:38.042468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d7bbc-e441-447d-b07b-facfa8839574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:38.043088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7bbc-e441-447d-b07b-facfa8839574 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:38.043362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7bbc-e441-447d-b07b-facfa8839574 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:38.043655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7bbc-e441-447d-b07b-facfa8839574 None admin] PATH_INFO: `/v3/projects/a122f8edfaf149248e7a575a66409992` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:38.044748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d7bbc-e441-447d-b07b-facfa8839574 None admin] RBAC: Authorizing `identity:get_project(project_id=a122f8edfaf149248e7a575a66409992)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:38.046163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d7bbc-e441-447d-b07b-facfa8839574 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:38.047860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 432/848] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:30:37 2022] GET /identity/v3/projects/a122f8edfaf149248e7a575a66409992 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:38.053863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:38.070098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:38.151713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856f979-b669-4835-a5fd-7b83ddd5f406 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:38.184199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6797b-ee2d-4231-9ed3-d2c105f4309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:38.184838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6797b-ee2d-4231-9ed3-d2c105f4309f None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:38.185111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6797b-ee2d-4231-9ed3-d2c105f4309f None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:38.185377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6797b-ee2d-4231-9ed3-d2c105f4309f None admin] PATH_INFO: `/v3/projects/a122f8edfaf149248e7a575a66409992/users/d35f1c163eb344658e2c75d1a876fa01/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:38.187500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6797b-ee2d-4231-9ed3-d2c105f4309f None admin] RBAC: Authorizing `identity:create_grant(project_id=a122f8edfaf149248e7a575a66409992, user_id=d35f1c163eb344658e2c75d1a876fa01, role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:38.188912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6797b-ee2d-4231-9ed3-d2c105f4309f None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:38.201737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 417/849] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:30:38 2022] PUT /identity/v3/projects/a122f8edfaf149248e7a575a66409992/users/d35f1c163eb344658e2c75d1a876fa01/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:30:40.309499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf072f0-ece6-4e7e-95fd-e4894b78b76a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:40.310028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf072f0-ece6-4e7e-95fd-e4894b78b76a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:40.310028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf072f0-ece6-4e7e-95fd-e4894b78b76a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:40.311382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 433/850] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:40.316591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:40.317035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:40.317127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:40.335015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:40.336205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:40.399470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 418/851] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:40.665588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf072f0-ece6-4e7e-95fd-e4894b78b76a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:40.684600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:40.764053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf072f0-ece6-4e7e-95fd-e4894b78b76a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:40.796793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:40.797402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:40.797678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:40.797943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] PATH_INFO: `/v3/roles/ade080f5f1cd46b89ad7b9088d1f1654` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:40.799308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] RBAC: Authorizing `identity:get_role(role_id=ade080f5f1cd46b89ad7b9088d1f1654)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:40.800764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:40.802059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 434/852] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:40 2022] GET /identity/v3/roles/ade080f5f1cd46b89ad7b9088d1f1654 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:40.807727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:40.810724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:40.904261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d0dd5-214e-400e-8faf-fb03db350e36 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:40.936348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:40.936908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:40.937689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:40.937968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] PATH_INFO: `/v3/users/d35f1c163eb344658e2c75d1a876fa01` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:40.939075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] RBAC: Authorizing `identity:get_user(user_id=d35f1c163eb344658e2c75d1a876fa01)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:40.940519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:40.942072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 419/853] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:40 2022] GET /identity/v3/users/d35f1c163eb344658e2c75d1a876fa01 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:40.947730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:41.029240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:41.043848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b18b-2744-4c7d-9f7f-33fb19484111 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:41.077002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:41.077597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:41.078414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:41.078698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:41.143281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:41.144753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:41.223075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:41.238564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 435/854] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:30:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:30:41.244526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:41.311174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:41.339721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633e64a-f825-495d-b9aa-57dbfcc1e43a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:41.371529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c34e74-2d60-4c0a-b6bb-b0e340876d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:41.372141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c34e74-2d60-4c0a-b6bb-b0e340876d84 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:41.372420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c34e74-2d60-4c0a-b6bb-b0e340876d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:41.372697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c34e74-2d60-4c0a-b6bb-b0e340876d84 None admin] PATH_INFO: `/v3/projects/a122f8edfaf149248e7a575a66409992` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:41.373789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c34e74-2d60-4c0a-b6bb-b0e340876d84 None admin] RBAC: Authorizing `identity:get_project(project_id=a122f8edfaf149248e7a575a66409992)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:41.375843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c34e74-2d60-4c0a-b6bb-b0e340876d84 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:41.377421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 420/855] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:30:41 2022] GET /identity/v3/projects/a122f8edfaf149248e7a575a66409992 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:41.383648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:41.480090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ba210-ab37-40c7-9936-f66d700b0ee2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:41.512309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:41.512899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:41.513176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:41.513443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:41.514158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] RBAC: Adding query filter params (role.id=ade080f5f1cd46b89ad7b9088d1f1654, scope.project.id=a122f8edfaf149248e7a575a66409992, user.id=d35f1c163eb344658e2c75d1a876fa01) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:41.514461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:41.515962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91443bde-21e1-4a74-b84e-19e9049d2b99 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:41.521421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 436/856] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:30:41 2022] GET /identity/v3/role_assignments?user.id=d35f1c163eb344658e2c75d1a876fa01&scope.project.id=a122f8edfaf149248e7a575a66409992&role.id=ade080f5f1cd46b89ad7b9088d1f1654 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:43.653507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120a81-8330-4f4e-885a-93f070a4bfd7 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:43.654047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120a81-8330-4f4e-885a-93f070a4bfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:43.654047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120a81-8330-4f4e-885a-93f070a4bfd7 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:43.655385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 421/857] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:43.661455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:43.661850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:43.661937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:43.679539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:43.680716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:43.724815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:43.744770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 437/858] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:44.014998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73120a81-8330-4f4e-885a-93f070a4bfd7 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:44.113163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73120a81-8330-4f4e-885a-93f070a4bfd7 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:44.145175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242515bd-937f-48ee-b364-cf377304ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:44.145782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242515bd-937f-48ee-b364-cf377304ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:44.146058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242515bd-937f-48ee-b364-cf377304ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:44.146326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242515bd-937f-48ee-b364-cf377304ac4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:44.147430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242515bd-937f-48ee-b364-cf377304ac4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:44.148860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242515bd-937f-48ee-b364-cf377304ac4d None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:44.150377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 422/859] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:44.156013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:44.250570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decea96c-eecb-4878-be14-62ef71caeaf2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:44.282740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc0525-5f1d-4a33-bf6b-7e7fea24c140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:44.283276 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc0525-5f1d-4a33-bf6b-7e7fea24c140 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:44.283581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc0525-5f1d-4a33-bf6b-7e7fea24c140 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:44.283852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc0525-5f1d-4a33-bf6b-7e7fea24c140 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:44.284503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc0525-5f1d-4a33-bf6b-7e7fea24c140 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:44.285918 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc0525-5f1d-4a33-bf6b-7e7fea24c140 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:44.311584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 438/860] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:44 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:46.467502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6023-e77b-4d88-a8e2-60374027d4ba None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:46.468017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6023-e77b-4d88-a8e2-60374027d4ba None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:46.468017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6023-e77b-4d88-a8e2-60374027d4ba None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:46.469417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 423/861] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:46.474583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:46.474987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:46.475128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:46.493389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:46.494561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:46.519768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:46.560037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 439/862] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:46.831385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6023-e77b-4d88-a8e2-60374027d4ba None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:46.841958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:46.928932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6023-e77b-4d88-a8e2-60374027d4ba None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:46.960830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b918b-2a1d-45c6-b3b1-e24a0760dc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:46.961436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b918b-2a1d-45c6-b3b1-e24a0760dc29 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:46.961708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b918b-2a1d-45c6-b3b1-e24a0760dc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:46.961971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b918b-2a1d-45c6-b3b1-e24a0760dc29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:46.963049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b918b-2a1d-45c6-b3b1-e24a0760dc29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:46.964479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b918b-2a1d-45c6-b3b1-e24a0760dc29 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:46.966000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 424/863] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:30:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:46.972198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:47.067675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627338ad-89d5-4e16-9922-34b98cd34963 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:47.214137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd12bc9-b2ed-4e9e-a5d6-c44739914c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:47.214730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd12bc9-b2ed-4e9e-a5d6-c44739914c76 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:47.215007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd12bc9-b2ed-4e9e-a5d6-c44739914c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:47.215273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd12bc9-b2ed-4e9e-a5d6-c44739914c76 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:47.215950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd12bc9-b2ed-4e9e-a5d6-c44739914c76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:47.217341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd12bc9-b2ed-4e9e-a5d6-c44739914c76 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:47.237458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 440/864] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:30:46 2022] POST /identity/v3/users => generated 286 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:49.365302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2b9f-cc43-4730-920a-162bfc929ed9 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:49.365871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2b9f-cc43-4730-920a-162bfc929ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:49.365871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2b9f-cc43-4730-920a-162bfc929ed9 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:49.367182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 425/865] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:49.372396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:49.372791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:49.372935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:49.390995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:49.392200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:49.404982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:49.461235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 441/866] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:49.729266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d2b9f-cc43-4730-920a-162bfc929ed9 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:49.804531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:49.828649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d2b9f-cc43-4730-920a-162bfc929ed9 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:49.861293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:49.861906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:49.862186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:49.862459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:49.872088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:49.876919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:49.877967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:30:49.879107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 426/867] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:49.883910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:49.977237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c42b0-28ef-46e6-bf4c-b643a067203a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:50.009542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:50.010070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:50.010339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:50.010600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:50.011252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:50.011575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:50.012922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:50.020670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 442/868] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:50.026413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:50.124188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14507f7-fde5-49d9-832a-8cb649cad879 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:50.157913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:50.158551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:50.158834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:50.159105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] PATH_INFO: `/v3/users/655edc8289c2438a836c11c81418dd72` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:50.161695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:50.172011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] RBAC: Authorizing `identity:get_user(user_id=655edc8289c2438a836c11c81418dd72)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:50.173435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:50.175001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 427/869] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:50 2022] GET /identity/v3/users/655edc8289c2438a836c11c81418dd72 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:50.180747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:50.192845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:50.275319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb3bd6-93d0-477c-a499-ede20530bc8a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:50.307063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efa213-3a68-4114-81fe-65bb3b9ccc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:50.307695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa213-3a68-4114-81fe-65bb3b9ccc04 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:50.307967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa213-3a68-4114-81fe-65bb3b9ccc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:50.308229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa213-3a68-4114-81fe-65bb3b9ccc04 None admin] PATH_INFO: `/v3/projects/436f9ad7504f4c28aeb053364a2954f1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:50.309301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efa213-3a68-4114-81fe-65bb3b9ccc04 None admin] RBAC: Authorizing `identity:get_project(project_id=436f9ad7504f4c28aeb053364a2954f1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:50.310783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efa213-3a68-4114-81fe-65bb3b9ccc04 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:50.312415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 443/870] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:30:50 2022] GET /identity/v3/projects/436f9ad7504f4c28aeb053364a2954f1 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:50.318242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:50.412911 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def232d-42e8-4e6a-ad93-155928621c08 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:50.445376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f32a01-c2b4-4660-8ad6-fe77a0b2fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:50.446017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32a01-c2b4-4660-8ad6-fe77a0b2fde4 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:50.446296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32a01-c2b4-4660-8ad6-fe77a0b2fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:50.446567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f32a01-c2b4-4660-8ad6-fe77a0b2fde4 None admin] PATH_INFO: `/v3/projects/436f9ad7504f4c28aeb053364a2954f1/users/655edc8289c2438a836c11c81418dd72/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:50.448449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32a01-c2b4-4660-8ad6-fe77a0b2fde4 None admin] RBAC: Authorizing `identity:create_grant(project_id=436f9ad7504f4c28aeb053364a2954f1, user_id=655edc8289c2438a836c11c81418dd72, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:50.449856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f32a01-c2b4-4660-8ad6-fe77a0b2fde4 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:50.462326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 428/871] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:30:50 2022] PUT /identity/v3/projects/436f9ad7504f4c28aeb053364a2954f1/users/655edc8289c2438a836c11c81418dd72/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:30:52.573107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84552e5b-f75d-4100-90af-f0273eff7a0e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:52.573625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84552e5b-f75d-4100-90af-f0273eff7a0e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:52.573625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84552e5b-f75d-4100-90af-f0273eff7a0e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:52.574995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 444/872] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:52.580258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:52.580660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:52.580792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:52.598127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:52.599291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:52.646044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:52.664157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 429/873] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 06:30:52.929065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84552e5b-f75d-4100-90af-f0273eff7a0e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:53.004507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:53.026159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84552e5b-f75d-4100-90af-f0273eff7a0e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:53.058769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:53.059435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:53.059716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:53.059983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:53.070065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:30:53.074837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:30:53.075900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:30:53.077093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 445/874] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:30:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:30:53.082441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:53.177079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc24eb7-a0d8-4c8e-bee5-1f0fdcb1ab50 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:53.209396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:53.209942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:53.210220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:53.210490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:53.211148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:53.211473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:53.212820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:53.221082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 430/875] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:53.226992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:53.322128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4484c4-72fb-411e-a48f-420ce824413c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:53.354432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:53.355034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:53.355312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:53.355610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] PATH_INFO: `/v3/users/655edc8289c2438a836c11c81418dd72` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:53.356696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] RBAC: Authorizing `identity:get_user(user_id=655edc8289c2438a836c11c81418dd72)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:53.358109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:53.359695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 446/876] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:30:53 2022] GET /identity/v3/users/655edc8289c2438a836c11c81418dd72 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:53.365350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:53.410285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:53.462021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0545aff-cb0c-48cc-bb21-457d9d8c6b54 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:53.493342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:53.493936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:53.494213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:53.494482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:53.557706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:53.559122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:53.601265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:53.652922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 431/877] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:30:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:30:53.659066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:53.668808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:53.756460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f7bb5-8b63-4dfe-aa69-e45f5e337dde None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:53.788769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6cf5a-542d-40d5-94a9-1363304da021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:53.789380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6cf5a-542d-40d5-94a9-1363304da021 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:53.789653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6cf5a-542d-40d5-94a9-1363304da021 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:53.789919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6cf5a-542d-40d5-94a9-1363304da021 None admin] PATH_INFO: `/v3/projects/436f9ad7504f4c28aeb053364a2954f1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:53.791008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6cf5a-542d-40d5-94a9-1363304da021 None admin] RBAC: Authorizing `identity:get_project(project_id=436f9ad7504f4c28aeb053364a2954f1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:53.792526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6cf5a-542d-40d5-94a9-1363304da021 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:53.794123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 447/878] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:30:53 2022] GET /identity/v3/projects/436f9ad7504f4c28aeb053364a2954f1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:53.800340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:53.896246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa44bc3-3eee-44b6-a7eb-2e11cc000108 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:53.928523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:53.929107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:53.929385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:53.929656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:53.930372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=436f9ad7504f4c28aeb053364a2954f1, user.id=655edc8289c2438a836c11c81418dd72) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:30:53.930678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:53.932123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c1f244-eff1-4266-a994-2ba20f247632 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:53.938159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 432/879] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:30:53 2022] GET /identity/v3/role_assignments?user.id=655edc8289c2438a836c11c81418dd72&scope.project.id=436f9ad7504f4c28aeb053364a2954f1&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:56.051490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad842068-3ee7-46be-8a41-2f8574a9fe90 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:56.052073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad842068-3ee7-46be-8a41-2f8574a9fe90 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:56.052073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad842068-3ee7-46be-8a41-2f8574a9fe90 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:56.053349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 448/880] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:56.058574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d13b0-1bd8-43a2-ac6a-f647d99a751c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:56.059011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d13b0-1bd8-43a2-ac6a-f647d99a751c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:56.059118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d13b0-1bd8-43a2-ac6a-f647d99a751c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:56.076716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-630d13b0-1bd8-43a2-ac6a-f647d99a751c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:56.078400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-630d13b0-1bd8-43a2-ac6a-f647d99a751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:56.081057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:56.142081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 433/881] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:56.407366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad842068-3ee7-46be-8a41-2f8574a9fe90 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:56.480168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:56.505256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad842068-3ee7-46be-8a41-2f8574a9fe90 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:56.537376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3ef2-eea3-4476-b368-ee67150beee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:56.537969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3ef2-eea3-4476-b368-ee67150beee4 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:56.538243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3ef2-eea3-4476-b368-ee67150beee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:56.538511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3ef2-eea3-4476-b368-ee67150beee4 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:56.539169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3ef2-eea3-4476-b368-ee67150beee4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:56.540548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3ef2-eea3-4476-b368-ee67150beee4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:56.555667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 449/882] 10.222.0.39 () {60 vars in 1138 bytes} [Tue Oct 11 06:30:56 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:30:58.703510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9361b6-3eb5-4f60-b3ff-b98aa447cb2f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:58.704012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9361b6-3eb5-4f60-b3ff-b98aa447cb2f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:58.704012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9361b6-3eb5-4f60-b3ff-b98aa447cb2f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:58.705328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 434/883] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:30:58.710474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:58.710846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:58.711024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:58.728716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:30:58.729895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:30:58.794093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 450/884] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:30:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:30:59.060390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9361b6-3eb5-4f60-b3ff-b98aa447cb2f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:59.158410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9361b6-3eb5-4f60-b3ff-b98aa447cb2f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:59.190164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b3939-c3c8-4768-a00b-23827637f1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:59.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b3939-c3c8-4768-a00b-23827637f1ad None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:59.191052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b3939-c3c8-4768-a00b-23827637f1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:59.191329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b3939-c3c8-4768-a00b-23827637f1ad None admin] PATH_INFO: `/v3/domains/469c78f7421f46dbbf4715aa52c30ff1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:59.199117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:59.202496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b3939-c3c8-4768-a00b-23827637f1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=469c78f7421f46dbbf4715aa52c30ff1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:59.204333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b3939-c3c8-4768-a00b-23827637f1ad None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:59.205926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 435/885] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Oct 11 06:30:59 2022] GET /identity/v3/domains/469c78f7421f46dbbf4715aa52c30ff1 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:30:59.211678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:30:59.270359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:30:59.307108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cef13-339d-4588-943f-b39905d9ec8f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:30:59.339606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf127be-e1bc-4f42-95d9-e469a722a6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:30:59.340148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf127be-e1bc-4f42-95d9-e469a722a6f0 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:30:59.340426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf127be-e1bc-4f42-95d9-e469a722a6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:30:59.340696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf127be-e1bc-4f42-95d9-e469a722a6f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:30:59.341351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf127be-e1bc-4f42-95d9-e469a722a6f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:30:59.343366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf127be-e1bc-4f42-95d9-e469a722a6f0 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:30:59.368123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 451/886] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:30:59 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:31:01.519551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9be310-831f-4f6a-af3e-abc6508918b1 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:01.520087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9be310-831f-4f6a-af3e-abc6508918b1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:01.520087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9be310-831f-4f6a-af3e-abc6508918b1 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:01.521447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 436/887] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:31:01.526733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:01.527145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:01.527284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:01.546071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:31:01.547255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:31:01.611161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 452/888] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:31:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:31:01.880947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9be310-831f-4f6a-af3e-abc6508918b1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:01.978603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9be310-831f-4f6a-af3e-abc6508918b1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:02.010583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f181a-a672-4cdb-8aeb-16f71ccc4df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:02.011188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f181a-a672-4cdb-8aeb-16f71ccc4df0 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:02.011485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f181a-a672-4cdb-8aeb-16f71ccc4df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:02.011755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f181a-a672-4cdb-8aeb-16f71ccc4df0 None admin] PATH_INFO: `/v3/domains/469c78f7421f46dbbf4715aa52c30ff1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:02.012838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f181a-a672-4cdb-8aeb-16f71ccc4df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=469c78f7421f46dbbf4715aa52c30ff1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:02.014258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f181a-a672-4cdb-8aeb-16f71ccc4df0 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:02.015805 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 437/889] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Oct 11 06:31:01 2022] GET /identity/v3/domains/469c78f7421f46dbbf4715aa52c30ff1 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:02.021341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:02.065885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:02.118025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4aca-c6ec-4765-a7ac-ff32ab62b740 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:02.150546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ed223-a9ee-4ef2-96ef-e202e210b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:02.151084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed223-a9ee-4ef2-96ef-e202e210b2c9 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:02.151364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed223-a9ee-4ef2-96ef-e202e210b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:02.151806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed223-a9ee-4ef2-96ef-e202e210b2c9 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:02.152460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ed223-a9ee-4ef2-96ef-e202e210b2c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:02.153888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ed223-a9ee-4ef2-96ef-e202e210b2c9 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:02.175466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 453/890] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:31:02 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:31:04.297761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c1fce-b7f5-4c7b-a0d5-457eef4990a9 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:04.298335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c1fce-b7f5-4c7b-a0d5-457eef4990a9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:04.298335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c1fce-b7f5-4c7b-a0d5-457eef4990a9 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:04.299660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 438/891] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:31:04.304688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:04.305098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:04.305235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:04.324280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:31:04.325457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:31:04.389122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 454/892] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:31:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:31:04.656200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c1fce-b7f5-4c7b-a0d5-457eef4990a9 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:04.720484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:04.752334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c1fce-b7f5-4c7b-a0d5-457eef4990a9 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:04.784218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:04.784825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:04.785098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:04.785364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:04.794842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:31:04.800099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:31:04.801213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:31:04.802283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 439/893] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:31:04.806965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:04.835948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:04.902260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2d90e-b577-493f-811c-6995693364e0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:04.935572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:04.936133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:04.936408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:04.936675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:04.937329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:31:04.937625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:04.938979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:04.946680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 455/894] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:04.952302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:05.047028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0703-a4d7-491d-bef9-77acf4cab252 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:05.071374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:05.079503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:05.080105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:05.080378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:05.080645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] PATH_INFO: `/v3/users/412be43f24be4b0ea5e2866ebd782830` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:05.093379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] RBAC: Authorizing `identity:get_user(user_id=412be43f24be4b0ea5e2866ebd782830)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:05.094787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:05.096370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 440/895] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:31:04 2022] GET /identity/v3/users/412be43f24be4b0ea5e2866ebd782830 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:05.102069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:05.194243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:05.201119 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753a64-0be0-4e41-bb34-10a662a5c15b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:05.234099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbaa1a-a865-450c-b562-068e7def6da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:05.234721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbaa1a-a865-450c-b562-068e7def6da2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:05.234997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbaa1a-a865-450c-b562-068e7def6da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:05.235263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbaa1a-a865-450c-b562-068e7def6da2 None admin] PATH_INFO: `/v3/projects/fdfda4886d8f48e8b07cb9385cb9876d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:05.236427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbaa1a-a865-450c-b562-068e7def6da2 None admin] RBAC: Authorizing `identity:get_project(project_id=fdfda4886d8f48e8b07cb9385cb9876d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:05.237845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbaa1a-a865-450c-b562-068e7def6da2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:05.239500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 456/896] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:31:05 2022] GET /identity/v3/projects/fdfda4886d8f48e8b07cb9385cb9876d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:05.245421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:05.339294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ad822-cc12-49f4-80d9-ff12e43307e6 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:05.371400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1496620-a5e1-4e05-be67-9544b79ee525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:05.372058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1496620-a5e1-4e05-be67-9544b79ee525 None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:05.372331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1496620-a5e1-4e05-be67-9544b79ee525 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:05.372595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1496620-a5e1-4e05-be67-9544b79ee525 None admin] PATH_INFO: `/v3/projects/fdfda4886d8f48e8b07cb9385cb9876d/users/412be43f24be4b0ea5e2866ebd782830/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:05.374423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1496620-a5e1-4e05-be67-9544b79ee525 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdfda4886d8f48e8b07cb9385cb9876d, user_id=412be43f24be4b0ea5e2866ebd782830, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:05.375847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1496620-a5e1-4e05-be67-9544b79ee525 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:05.388086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 441/897] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:31:05 2022] PUT /identity/v3/projects/fdfda4886d8f48e8b07cb9385cb9876d/users/412be43f24be4b0ea5e2866ebd782830/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:31:07.484312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7136c8-a4d4-4e8f-9fa6-4bb7f32862ef None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:07.484819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7136c8-a4d4-4e8f-9fa6-4bb7f32862ef None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:07.484927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7136c8-a4d4-4e8f-9fa6-4bb7f32862ef None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:07.486230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 457/898] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:31:07.491656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:07.492037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:07.492212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:07.510340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:31:07.511560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:31:07.553893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:07.578559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 442/899] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:31:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:31:07.849211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7136c8-a4d4-4e8f-9fa6-4bb7f32862ef None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:07.948158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7136c8-a4d4-4e8f-9fa6-4bb7f32862ef None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:07.980458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:07.981072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:07.981349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:07.981617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:07.992505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:31:07.997271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:31:07.998323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:31:07.999270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 458/900] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:31:08.004359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:08.097348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb224f-ca77-49cf-9eda-2651b4d669c2 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:08.129548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:08.130081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:08.130359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:08.130627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:08.131281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:31:08.131608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:08.132948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:08.141088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 443/901] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:31:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:08.146835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:08.171356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:08.243430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a17d2-2920-441e-8443-22fb5206e951 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:08.275352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:08.275990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:08.276265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:08.276535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] PATH_INFO: `/v3/users/412be43f24be4b0ea5e2866ebd782830` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:08.277636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] RBAC: Authorizing `identity:get_user(user_id=412be43f24be4b0ea5e2866ebd782830)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:08.279054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:08.281272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 459/902] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Oct 11 06:31:08 2022] GET /identity/v3/users/412be43f24be4b0ea5e2866ebd782830 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:08.287212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:08.325152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:08.382454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0278c-c633-4c33-b24a-f584d406e60b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:08.416417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:08.417029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:08.417309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:08.417583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:08.481293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:08.482720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:08.518534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:08.575945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 444/903] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:31:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:31:08.581808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:08.670712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:08.680419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ab0e-5c98-4644-b0f0-dac90b7b5f11 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:08.712174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2049a-e0e1-4a5d-9b8d-812f3e682d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:08.712791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2049a-e0e1-4a5d-9b8d-812f3e682d35 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:08.713070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2049a-e0e1-4a5d-9b8d-812f3e682d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:08.713342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2049a-e0e1-4a5d-9b8d-812f3e682d35 None admin] PATH_INFO: `/v3/projects/fdfda4886d8f48e8b07cb9385cb9876d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:08.714441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2049a-e0e1-4a5d-9b8d-812f3e682d35 None admin] RBAC: Authorizing `identity:get_project(project_id=fdfda4886d8f48e8b07cb9385cb9876d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:08.715899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2049a-e0e1-4a5d-9b8d-812f3e682d35 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:08.718009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 460/904] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Oct 11 06:31:08 2022] GET /identity/v3/projects/fdfda4886d8f48e8b07cb9385cb9876d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:31:08.724165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:31:08.817413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2662c38-978a-48a5-a754-72c0d89eea6a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:31:08.849898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:31:08.850479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:31:08.850754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:31:08.851023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:31:08.851761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=fdfda4886d8f48e8b07cb9385cb9876d, user.id=412be43f24be4b0ea5e2866ebd782830) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:31:08.852064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:31:08.853471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023dbd3-f088-4475-916c-61fe030592ce None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:31:08.858949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 445/905] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:31:08 2022] GET /identity/v3/role_assignments?user.id=412be43f24be4b0ea5e2866ebd782830&scope.project.id=fdfda4886d8f48e8b07cb9385cb9876d&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:31:18.708135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=21 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:31:18.856499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:03.760345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b4c30-edef-4179-bed9-9724cb2bb932 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:03.760946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b4c30-edef-4179-bed9-9724cb2bb932 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:03.760946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b4c30-edef-4179-bed9-9724cb2bb932 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:03.762513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 461/906] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:03.768551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:03.769060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:03.769146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:03.787877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:03.789074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:03.882102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 446/907] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:04.168310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b4c30-edef-4179-bed9-9724cb2bb932 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:04.300014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b4c30-edef-4179-bed9-9724cb2bb932 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:04.350668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:04.351302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:04.351607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:04.351876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:04.365702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:04.367542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:04.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:04.387592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:04.388719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:04.388719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:04.388719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:04.388719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 462/908] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:04.396130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:04.507371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613a7e8-06ea-45b0-b098-016590aadcc1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:04.548022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:04.554701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:04.555515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:04.555766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:04.556016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:04.556666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:04.556967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:04.558356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a853c1-4c69-4da9-a529-07fe195c76b6 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:04.570551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 447/909] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:04.576236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:04.632020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:04.689802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a24b03-102f-48bb-a1da-317a0b90b275 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:04.728825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123db7bc-c31f-44b9-a116-da28179eb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:04.729928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db7bc-c31f-44b9-a116-da28179eb858 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:04.730451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db7bc-c31f-44b9-a116-da28179eb858 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:04.730982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db7bc-c31f-44b9-a116-da28179eb858 None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:04.731904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123db7bc-c31f-44b9-a116-da28179eb858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:04.734768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123db7bc-c31f-44b9-a116-da28179eb858 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:04.767595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 463/910] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Oct 11 06:32:04 2022] POST /identity/v3/users => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:32:07.083957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc6e9-2631-41d9-a5c7-03c58a7d223c None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:07.084493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc6e9-2631-41d9-a5c7-03c58a7d223c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:07.084493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc6e9-2631-41d9-a5c7-03c58a7d223c None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:07.085826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 448/911] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:07.092437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:07.092894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:07.093005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:07.113785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:07.115620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:07.203992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 464/912] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:07.502921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc6e9-2631-41d9-a5c7-03c58a7d223c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:07.626437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc6e9-2631-41d9-a5c7-03c58a7d223c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:07.667070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:07.667695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:07.668072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:07.668386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:07.681084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:32:07.684469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:07.685679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:32:07.686707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:07.694293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 449/913] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:32:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:07.694457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:07.742547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:07.835485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f7325-388a-49a2-af74-6b7a7bb8bcfb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:07.892535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:07.893359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:07.893614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:07.894004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:07.894793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:07.895215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:07.896967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:07.908295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 465/914] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:32:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:07.914671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:07.972796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:08.030074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61283777-9ffd-43b7-b768-5550a9c6d080 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:08.069542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:08.070230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:08.070562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:08.070796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:08.083007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:08.085377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:08.104595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:08.110818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:08.111918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:08.111918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:08.111918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:08.111918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 450/915] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:08.112204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:08.233880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:08.237297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999369b2-835b-43f0-8109-af5354ffa6c8 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:08.274242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:08.274854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:08.275136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:08.275407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:08.276616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:08.276916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:08.278357 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:08.294775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 466/916] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:08.300751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:08.413269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b3ea-2d3b-4b49-9f81-137e352c3b1b None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:08.415232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:08.450780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:08.451394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:08.453061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:08.453514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:08.461534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:08.462937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:32:08.474906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:08.476143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:08.477352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:08.481020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 451/917] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:08.482197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:08.600616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66713f07-4ebf-4d44-9e70-69b21f413005 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:08.635976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:08.636629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:08.636913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:08.637136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:08.637880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:08.639676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:08.641152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:08.667439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 467/918] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Oct 11 06:32:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:08.673384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:08.782527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546f6a8-10f2-4849-bc53-9de59afb6abc None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:08.821815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:08.821815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:08.821815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:08.821815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:08.835744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:08.837517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:08.854936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:08.856133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:08.857053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:08.857053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:08.857053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:08.857053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 452/919] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:08.862198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:08.884645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:08.991523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9cab-b907-48e5-a305-206ff70a733d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:09.028998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:09.029855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:09.030168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:09.030395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:09.031025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:09.031316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:09.032694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:09.045463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 468/920] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:09.051088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:09.067821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:09.158275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a900d-fa14-44f1-9661-6c33f30dafa5 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:09.192971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:09.193598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:09.193876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:09.194144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:09.200815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:09.202252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:32:09.214134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:09.215260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:09.216381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:09.217685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 453/921] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:09.223726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:09.309967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:09.340555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfc0b1-c655-4c5e-863d-562e77d37932 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:09.379002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:09.379653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:09.379959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:09.380197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:09.380927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:09.381224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:09.382641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f17ddc-602a-4ff2-bebf-d864b8ce4347 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:09.396455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 469/922] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:09.404039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:09.489181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:09.518389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad2b99-a707-4221-b636-9b17390c4f41 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:09.559228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118a9f2-3540-4419-a16e-0b6a6cd8679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:09.560222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118a9f2-3540-4419-a16e-0b6a6cd8679e None admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:09.560595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118a9f2-3540-4419-a16e-0b6a6cd8679e None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:09.560994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118a9f2-3540-4419-a16e-0b6a6cd8679e None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/8d6eb8eb03314f6aa7fe0ba171babf57/roles/d5ab3f6344b74243b09de42facc82f8d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:09.579301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118a9f2-3540-4419-a16e-0b6a6cd8679e None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=8d6eb8eb03314f6aa7fe0ba171babf57, role_id=d5ab3f6344b74243b09de42facc82f8d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:09.581038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118a9f2-3540-4419-a16e-0b6a6cd8679e None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:09.599683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 454/923] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:32:09 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/8d6eb8eb03314f6aa7fe0ba171babf57/roles/d5ab3f6344b74243b09de42facc82f8d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:32:11.877168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725498b1-587e-47c5-8a7b-492eb2a495ed None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:11.877775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725498b1-587e-47c5-8a7b-492eb2a495ed None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:11.877775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725498b1-587e-47c5-8a7b-492eb2a495ed None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:11.879143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 470/924] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:11.885588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:11.886344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:11.886344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:11.905462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:11.906762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:11.979700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 455/925] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:12.272120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725498b1-587e-47c5-8a7b-492eb2a495ed None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:12.379920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725498b1-587e-47c5-8a7b-492eb2a495ed None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:12.420705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:12.421548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:12.421986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:12.422217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:12.429478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:12.435111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:32:12.441003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:12.442222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:12.443070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 471/926] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Oct 11 06:32:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:12.449169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:12.509807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:12.559222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2a46-7168-4ec5-8079-5ffe77e843de None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:12.594785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:12.595396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:12.595758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:12.596028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:12.596766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] RBAC: Adding query filter params (name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:12.597144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:12.598492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:12.607910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 456/927] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Oct 11 06:32:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:12.613812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:12.722121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec6cdf-2059-4190-8979-1b02196d9b6d None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:12.757702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:12.758331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:12.758630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:12.758887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:12.770370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:12.772100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:12.789642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:12.790853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:12.791948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:12.791948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:12.791948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:12.791948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 472/928] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:12.796812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:12.914874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb81005-e8b0-44c2-a8b8-52530f53fa14 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:12.933638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:12.955017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:12.955669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:12.955945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:12.956221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:12.956852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:12.957227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:12.958579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:12.970814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 457/929] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:12.977254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:13.029544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:13.081031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e742e2a-cc11-44de-bc93-2be85315f758 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:13.116534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:13.117555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:13.118077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:13.118797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:13.132378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:13.134389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:32:13.153124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:13.154556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:13.155560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 473/930] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:32:13.160508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:13.265877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35572a1e-bfed-43b8-8372-aec7dbc940ae None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:13.302937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:13.303682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:13.303959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:13.304272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:13.305097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:13.305477 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:13.307539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:13.326786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 458/931] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Oct 11 06:32:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:13.332765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:13.440842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84743c7e-1a35-41f5-b5ad-c6bf37c9e4ea None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:13.453767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:13.481497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:13.482211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:13.482545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:13.482817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:13.554871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:13.556263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:13.665759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 474/932] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:32:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 06:32:13.671492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:13.705892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:13.777582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95ef34-d3af-4a74-a622-f746da6b91e3 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:13.811930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:13.812563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:13.812843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:13.813116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:13.824544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:13.826115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:13.840935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:13.842031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:13.843049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:13.843049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:13.843049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:13.843049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 459/933] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:13.848342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:13.860956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:13.956488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf26c51-cebe-4c3b-af86-0f5cdf4507a6 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:13.994781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:13.995525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:13.995679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:13.995946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:13.996589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:13.996887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:13.998241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:14.010530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 475/934] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:14.016334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:14.109261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:14.128008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bf1ff-d3ad-4a9d-9f15-e6479db50cba None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:14.167203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:14.167874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:14.168150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:14.168423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:14.175551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:14.176990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:32:14.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:14.189903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:14.190777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 460/935] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:32:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:14.197062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:14.272274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:14.310910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f390-7540-4ac1-819c-20d5d00cdeb1 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:14.346940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:14.347663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:14.348014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:14.348256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:14.348919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:14.349221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:14.350654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243ee1e-c718-4ae6-a287-6a7ed9f4cef6 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:14.364478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 476/936] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:32:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:14.371857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:14.485331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36d5fe-09a9-41da-bcd4-8382c616d185 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:14.522029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:14.522678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:14.522968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:14.523240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:14.524011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] RBAC: Adding query filter params (role.id=d5ab3f6344b74243b09de42facc82f8d, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=8d6eb8eb03314f6aa7fe0ba171babf57) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:14.524317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:14.526473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209cf89-2903-448f-a5ca-28bd7ff2eb40 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:14.532856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 461/937] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:32:14 2022] GET /identity/v3/role_assignments?user.id=8d6eb8eb03314f6aa7fe0ba171babf57&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=d5ab3f6344b74243b09de42facc82f8d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:16.764485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723ff9-e88b-49e5-854b-d5830ffa53d6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:16.765100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723ff9-e88b-49e5-854b-d5830ffa53d6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:16.765100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723ff9-e88b-49e5-854b-d5830ffa53d6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:16.766288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 477/938] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:16.772118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:16.772544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:16.772787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:16.792820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:16.794298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:16.797233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:16.866974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 462/939] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:17.166049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723ff9-e88b-49e5-854b-d5830ffa53d6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:17.283235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723ff9-e88b-49e5-854b-d5830ffa53d6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:17.312178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:17.318803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:17.319676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:17.320150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:17.320421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:17.331884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:32:17.336863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:17.340202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:17.341320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 478/940] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:17.345025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:17.447878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d083e-f493-431b-8242-4eed50ba3f15 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:17.484634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:17.485376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:17.485685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:17.485962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:17.486753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:17.487062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:17.488669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:17.498370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 463/941] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:17.504197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:17.605529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54615-0b17-4fe5-9c6f-640a4e8a01c4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:17.642767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:17.643441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:17.643833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:17.644106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:17.654436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:17.655992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:17.672946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:17.674111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:17.675237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:17.675237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:17.675237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:17.675237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 479/942] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:32:17.680077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:17.683777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:17.792222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b10a89-b182-4b7e-879a-975ad7804314 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:17.826866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:17.827524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:17.827802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:17.828069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:17.828702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:17.828999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:17.830352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:17.841838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 464/943] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:32:17.848285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:17.896993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:17.966382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4ace5-4bd9-4bd7-91c7-eedffb76c9d4 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:18.005194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:18.005830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:18.006148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:18.006422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:18.016153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:18.017642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:32:18.029410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:18.035059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:18.036280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 480/944] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:32:18.037005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:18.106184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:18.150220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc597b-3a04-4081-a262-8bbc57bd49dd None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:18.193649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:18.194249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:18.194525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:18.194794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:18.195550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:18.195879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:18.197377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:18.215128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 465/945] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Oct 11 06:32:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:18.221008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:18.328755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1904d-d0c3-4deb-82eb-a556fd021717 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:18.330362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:18.364447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:18.365026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:18.365301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:18.365566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:18.378048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:18.379635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:18.394080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:18.395202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:18.396280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:18.396280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:18.396280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:18.396280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 481/946] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:18.402050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:18.516206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0467d-2a14-4cec-b9cb-514f9e155dca None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:18.540941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:18.557125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:18.557820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:18.558135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:18.558451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:18.559185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:18.559534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:18.561089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:18.574214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 466/947] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:18.580138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:18.695138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70e2a7-5a3c-4ed0-a559-48e9b87d54d7 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:18.738245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:18.739163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:18.739677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:18.739987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:18.748152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:18.749734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:32:18.762001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:18.763875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:18.765947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:18.768166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 482/948] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:18.769170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:18.880088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207919b-bbaf-468e-ab78-211ba6022286 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:18.917438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:18.918094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:18.918432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:18.918685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:18.919498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:18.919806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:18.921228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6fd29-a0b3-42f4-807d-2f6a2b9acd5c None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:18.934219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 467/949] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:32:18.940458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:18.957543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:19.057473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2863544-28d2-4728-b7b2-55563449fc5c None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:19.093464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9b01d-52c8-41ae-8303-60be970f046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:19.094238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b01d-52c8-41ae-8303-60be970f046d None admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:19.094515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b01d-52c8-41ae-8303-60be970f046d None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:19.094813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b01d-52c8-41ae-8303-60be970f046d None admin] PATH_INFO: `/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/8d6eb8eb03314f6aa7fe0ba171babf57/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:19.096753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b01d-52c8-41ae-8303-60be970f046d None admin] RBAC: Authorizing `identity:create_grant(project_id=89e92b29982040fe9ee65a3a0bae8af8, user_id=8d6eb8eb03314f6aa7fe0ba171babf57, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:19.098429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b01d-52c8-41ae-8303-60be970f046d None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:19.111678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 483/950] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Oct 11 06:32:18 2022] PUT /identity/v3/projects/89e92b29982040fe9ee65a3a0bae8af8/users/8d6eb8eb03314f6aa7fe0ba171babf57/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:32:21.508178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d13f2-32a9-4931-a9a8-dfe4deef938d None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:21.508733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d13f2-32a9-4931-a9a8-dfe4deef938d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:21.508733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d13f2-32a9-4931-a9a8-dfe4deef938d None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:21.510744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 468/951] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:21.516274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:21.516836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:21.516836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:21.535781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:21.536976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:21.612330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 484/952] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:21.893824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d13f2-32a9-4931-a9a8-dfe4deef938d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:21.938923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:22.015858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d13f2-32a9-4931-a9a8-dfe4deef938d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:22.053629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:22.054292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:22.054573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:22.054847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:22.067028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 06:32:22.073412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:22.074549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:22.075373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 469/953] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Oct 11 06:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:22.083626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:22.087058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:22.210144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42294091-c26f-44e7-a5f9-3b7944aab1af None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:22.249537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:22.250151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:22.250428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:22.250698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:22.251371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:22.251696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:22.253071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:22.262522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 485/954] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Oct 11 06:32:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:22.268859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:22.381341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:22.384831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400afc90-d506-4b81-9231-46ffeb24ac76 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:22.428858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:22.429707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:22.430007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:22.430431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:22.444059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:22.445891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:22.462531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:22.463644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:22.464566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:22.464566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:22.464566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:22.464566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 470/955] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:22.469955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:22.542705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:22.579909 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c8e6-7cd8-4668-bd08-942f1208cbb2 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:22.616017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:22.616619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:22.616893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:22.617160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:22.617787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:22.618082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:22.619478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:22.631242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 486/956] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:32:22.638054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:22.745231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e7189-a798-4b89-9ab2-43bda17e7884 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:22.780404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:22.781041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:22.781318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:22.781587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:22.789537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:22.791018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 06:32:22.802332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:22.808873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:22.809987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 471/957] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:22.810579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:22.912429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec5310-eee4-41d0-8893-7f9f6f8bb7d5 None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:22.939039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:22.948762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:22.949376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:22.949648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:22.949914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:22.950651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:22.950949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:22.952442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:22.971450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 487/958] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Oct 11 06:32:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:22.977344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:22.992988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:23.082119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ee0e-bcec-47e7-a094-9db6564d1f8a None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:23.116740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:23.117490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:23.117846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:23.118181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:23.201503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:23.202947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:23.216591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:23.306909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 472/959] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Oct 11 06:32:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 11 06:32:23.312730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:23.414141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f9d3-0b45-45de-bd84-d8d77856f41b None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:23.448396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:23.449018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:23.449298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:23.449570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:23.459278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:23.460749 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 06:32:23.475274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:23.476359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 06:32:23.477604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 06:32:23.477604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 06:32:23.477604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:23.477604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 488/960] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:23.482698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:23.577350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fb4ab-f9c2-4e53-9cb8-c58bd0be29ea None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:23.590353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:23.611592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:23.612189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:23.612464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:23.612733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:23.613375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:23.613670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:23.615011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:23.625710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 473/961] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Oct 11 06:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:23.631281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:23.666763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:23.729224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd88da7-c4d0-4562-bc2c-d376b245980a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:23.761275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:23.761889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:23.762166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:23.762438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:23.768920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:23.770354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 06:32:23.782881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:23.783894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: ERROR keystone.server.flask.application Oct 11 06:32:23.784952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 489/962] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Oct 11 06:32:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:23.789729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:23.884129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38984a77-57a1-4128-902c-a4ec5701f4f7 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:23.915094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:23.916207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:23.916481 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:23.916744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:23.917466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:23.917717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:23.919121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0660c26-4910-47b9-9537-e8fe1466bbcd None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:23.930316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 474/963] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Oct 11 06:32:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:23.936745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:24.034376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:24.040258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41988c-6d4d-4350-8ecd-5e014744396a None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:24.072649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:24.073231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:24.073503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:24.073767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:24.074476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] RBAC: Adding query filter params (role.id=287ae34730df480e9b5436bfab5d48bd, scope.project.id=89e92b29982040fe9ee65a3a0bae8af8, user.id=8d6eb8eb03314f6aa7fe0ba171babf57) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:24.074777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:24.076214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420cc5f-f1cf-48b0-80f8-d0573e23b3e8 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:24.081753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 490/964] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Oct 11 06:32:23 2022] GET /identity/v3/role_assignments?user.id=8d6eb8eb03314f6aa7fe0ba171babf57&scope.project.id=89e92b29982040fe9ee65a3a0bae8af8&role.id=287ae34730df480e9b5436bfab5d48bd => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:26.210050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080c6f5-37f0-46f1-bebc-cdc66cfca0b3 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:26.210586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080c6f5-37f0-46f1-bebc-cdc66cfca0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:26.210586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080c6f5-37f0-46f1-bebc-cdc66cfca0b3 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:26.211964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 475/965] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:26.217622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:26.218019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:26.218165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:26.236292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:26.237451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:26.301615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 491/966] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:26.566248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080c6f5-37f0-46f1-bebc-cdc66cfca0b3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:26.597432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:26.664432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080c6f5-37f0-46f1-bebc-cdc66cfca0b3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:26.696536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:26.697148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:26.697425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:26.697695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:26.698268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:26.699640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:26.708078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:26.709278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:26.710139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 476/967] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:32:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:26.715552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:26.800800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:26.810809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd688a73-2019-4f1e-9c30-d5c80eae76d7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:26.844114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:26.844649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:26.844923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:26.845190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:26.845808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:26.846105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:26.847485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4af4-2456-4641-8cb9-093894fb8b30 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:26.852170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 492/968] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Oct 11 06:32:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:26.857953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:26.949199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:26.955528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8512c-c95e-4f18-8de1-a9d96ccf3087 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:26.987784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:26.988378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:26.988657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:26.988928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] PATH_INFO: `/v3/services` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:26.989550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:26.989852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:26.991195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa783f04-0bf5-4c05-a7d3-b73874af9250 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:26.995479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 477/969] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Oct 11 06:32:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:29.116402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a4bf9-299e-428f-bc9b-d0c7cc8a7455 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:29.116994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a4bf9-299e-428f-bc9b-d0c7cc8a7455 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:29.116994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a4bf9-299e-428f-bc9b-d0c7cc8a7455 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:29.118277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 493/970] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:29.123514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d6343-aba6-460b-9033-d30b0f2609fd None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:29.123968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d6343-aba6-460b-9033-d30b0f2609fd None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:29.124066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d6343-aba6-460b-9033-d30b0f2609fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:29.142090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-437d6343-aba6-460b-9033-d30b0f2609fd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:29.143264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-437d6343-aba6-460b-9033-d30b0f2609fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:29.206186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 478/971] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:29.474528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a4bf9-299e-428f-bc9b-d0c7cc8a7455 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:29.572553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a4bf9-299e-428f-bc9b-d0c7cc8a7455 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:29.604439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6bdb4-6371-4841-ba39-54d35f871449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:29.605028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6bdb4-6371-4841-ba39-54d35f871449 None admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:29.605301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6bdb4-6371-4841-ba39-54d35f871449 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:29.605570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6bdb4-6371-4841-ba39-54d35f871449 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:29.606217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6bdb4-6371-4841-ba39-54d35f871449 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:29.607586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6bdb4-6371-4841-ba39-54d35f871449 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:29.621960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 494/972] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Oct 11 06:32:29 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:32:31.745040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b48e-afe2-4241-80fa-cbbc11cbc28e None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:31.745550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b48e-afe2-4241-80fa-cbbc11cbc28e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:31.745550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b48e-afe2-4241-80fa-cbbc11cbc28e None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:31.746817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 479/973] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:31.751922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:31.752294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:31.752472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:31.770025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:31.771198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:31.780884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:31.842767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 495/974] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:32.111396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2b48e-afe2-4241-80fa-cbbc11cbc28e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:32.205168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:32.216074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2b48e-afe2-4241-80fa-cbbc11cbc28e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:32.247659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:32.248272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:32.248547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:32.248814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:32.249384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:32.250718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:32.259674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:32.260903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:32.261793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 480/975] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:32:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 06:32:32.266757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:32.360042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95d9bb-bc2d-4f64-bff5-5e5559f16081 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:32.392673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:32.393203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:32.393475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:32.393740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:32.394355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] RBAC: Adding query filter params (name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:32.394650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:32.396026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:32.401600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 496/976] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Oct 11 06:32:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:32.407745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:32.501174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c68665-774d-4609-89f4-3187ac4fa800 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:32.532934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:32.533511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:32.533785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:32.534049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:32.534729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] RBAC: Adding query filter params (interface=public, service_id=e8c1bdf41a354355a4cf3132247754d7, region_id=RegionOne) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:32.535028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:32.536391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8fd25b-10ab-43a3-843a-b9ad4f9f6763 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:32.541106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 481/977] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Oct 11 06:32:32 2022] GET /identity/v3/endpoints?service_id=e8c1bdf41a354355a4cf3132247754d7&interface=public®ion_id=RegionOne => generated 194 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:32.546733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:32.550388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:32.644454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa10d-89a8-4146-8581-313b110e59ec None admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:32.676994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459ced4-aca3-4e71-98f8-c33ce3e32061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:32.677596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459ced4-aca3-4e71-98f8-c33ce3e32061 None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:32.677874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459ced4-aca3-4e71-98f8-c33ce3e32061 None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:32.678143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459ced4-aca3-4e71-98f8-c33ce3e32061 None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:32.678741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459ced4-aca3-4e71-98f8-c33ce3e32061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:32.680128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459ced4-aca3-4e71-98f8-c33ce3e32061 None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:32.687312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 497/978] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Oct 11 06:32:32 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:32:34.791752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59138b5-3491-4850-b132-de26d6b92da3 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:34.792265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59138b5-3491-4850-b132-de26d6b92da3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:34.792265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59138b5-3491-4850-b132-de26d6b92da3 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:34.793543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 482/979] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:32:34.798597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:34.799032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:34.799132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:34.816647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:32:34.817837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:32:34.876500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:34.882065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 498/980] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:32:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 06:32:35.154888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59138b5-3491-4850-b132-de26d6b92da3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:35.177726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:35.258113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59138b5-3491-4850-b132-de26d6b92da3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:35.291102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:35.291740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:35.292018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:35.292288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:35.292858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:35.294197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 06:32:35.302660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 06:32:35.303812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 06:32:35.304632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 483/981] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Oct 11 06:32:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 06:32:35.309727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:35.404579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48f0a-dca8-4e72-a324-dfe815ce9f8d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:35.438526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:35.439126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:35.439406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:35.439707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] PATH_INFO: `/v3/services` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:35.440335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] RBAC: Adding query filter params (name=placement) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:32:35.440635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:35.442005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5043b-cb83-4eae-88b7-1817d26e65de None admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:35.447604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 499/982] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Oct 11 06:32:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:32:35.453237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:32:35.531313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:35.548773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b13df9-8340-4f33-bf28-9c15ecac7623 None admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:32:35.581187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e54f3-7537-4e8f-a5eb-ccfb7538d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:32:35.581780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e54f3-7537-4e8f-a5eb-ccfb7538d813 None admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:32:35.582061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e54f3-7537-4e8f-a5eb-ccfb7538d813 None admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:32:35.582335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e54f3-7537-4e8f-a5eb-ccfb7538d813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:32:35.582995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e54f3-7537-4e8f-a5eb-ccfb7538d813 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:32:35.584367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e54f3-7537-4e8f-a5eb-ccfb7538d813 None admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:32:35.602726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 484/983] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:32:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:32:45.447325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=69 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:32:45.598587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:38.959518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b52e1-8c3e-477e-a24f-24c359163c87 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:38.959990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b52e1-8c3e-477e-a24f-24c359163c87 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:38.960141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b52e1-8c3e-477e-a24f-24c359163c87 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:38.961397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 500/984] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:38.967267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b4e3-8120-4bf9-b5c0-01b93aa76ecc None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:38.967695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b4e3-8120-4bf9-b5c0-01b93aa76ecc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:38.967944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b4e3-8120-4bf9-b5c0-01b93aa76ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:39.055267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-78e6b4e3-8120-4bf9-b5c0-01b93aa76ecc None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:39.056464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-78e6b4e3-8120-4bf9-b5c0-01b93aa76ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:39.189107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 485/985] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:39.484354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246fef9-89fc-4e34-ba0a-8b78f0f123e6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:39.484744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246fef9-89fc-4e34-ba0a-8b78f0f123e6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:39.485148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246fef9-89fc-4e34-ba0a-8b78f0f123e6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:39.486854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 501/986] 10.222.0.39 () {54 vars in 964 bytes} [Tue Oct 11 06:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:39.493433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564918-23df-48d6-ac8b-7983820d874b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:39.493799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564918-23df-48d6-ac8b-7983820d874b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:39.494135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564918-23df-48d6-ac8b-7983820d874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:39.585245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-57564918-23df-48d6-ac8b-7983820d874b None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:39.586463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-57564918-23df-48d6-ac8b-7983820d874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:39.609671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:39.672991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 486/987] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Oct 11 06:33:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:39.682798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246fef9-89fc-4e34-ba0a-8b78f0f123e6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:39.684649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f246fef9-89fc-4e34-ba0a-8b78f0f123e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:39.797106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246fef9-89fc-4e34-ba0a-8b78f0f123e6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:39.876284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:39.877000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:39.877286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:39.877681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:39.879934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:39.943369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:40.003560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:40.005163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc053df4-e3b1-4bfa-afd7-67042c0ea773 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:40.024408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 502/988] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:33:42.461341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c76d-3bd4-4e33-b1a4-cf6bee98c909 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:42.461859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c76d-3bd4-4e33-b1a4-cf6bee98c909 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:42.461966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c76d-3bd4-4e33-b1a4-cf6bee98c909 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:42.463309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 487/989] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:42.468423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e989-932f-4c7d-b93e-1a799ae25327 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:42.468794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e989-932f-4c7d-b93e-1a799ae25327 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:42.468963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e989-932f-4c7d-b93e-1a799ae25327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:42.538152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b557e989-932f-4c7d-b93e-1a799ae25327 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:42.539256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b557e989-932f-4c7d-b93e-1a799ae25327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:42.632515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 503/990] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:42.910753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85c76d-3bd4-4e33-b1a4-cf6bee98c909 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:42.912407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fa85c76d-3bd4-4e33-b1a4-cf6bee98c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:43.017861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85c76d-3bd4-4e33-b1a4-cf6bee98c909 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:43.088218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:43.090834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:43.091691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:43.092162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:43.092649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:43.094258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:43.201958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:43.203294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c988289-d5d5-46ae-98d9-7f4541f0f8cb service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:43.221281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 488/991] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:33:45.659315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c92ca-1968-4bd0-9f47-39c19295a41a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:45.660414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c92ca-1968-4bd0-9f47-39c19295a41a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:45.660414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c92ca-1968-4bd0-9f47-39c19295a41a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:45.661748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 504/992] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:45.667052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef0c3-cc14-4310-a90e-07ff198cc34b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:45.667452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef0c3-cc14-4310-a90e-07ff198cc34b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:45.667631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef0c3-cc14-4310-a90e-07ff198cc34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:45.732550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-789ef0c3-cc14-4310-a90e-07ff198cc34b None None] MFA Rules not processed for user `291c9818263642f080c993ec53610ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:45.733755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-789ef0c3-cc14-4310-a90e-07ff198cc34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:45.783948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:45.827374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 489/993] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:46.111428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c92ca-1968-4bd0-9f47-39c19295a41a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:46.112957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-eb1c92ca-1968-4bd0-9f47-39c19295a41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:46.122606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:46.215506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c92ca-1968-4bd0-9f47-39c19295a41a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:46.286034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:46.286628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:46.286901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:46.287162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:46.288755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:46.354549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:46.399852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:46.401466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424b115-7c6c-44d7-892a-7d62e46478ee service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:46.419824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 505/994] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:33:49.062698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e953-1a09-49e0-b81e-ab005495e222 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:49.063236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e953-1a09-49e0-b81e-ab005495e222 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:49.063236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e953-1a09-49e0-b81e-ab005495e222 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:49.064627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 490/995] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:49.069806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186638-655e-42f8-8bea-7ffa76b502b1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:49.070239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186638-655e-42f8-8bea-7ffa76b502b1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:49.070360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186638-655e-42f8-8bea-7ffa76b502b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:49.134551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-85186638-655e-42f8-8bea-7ffa76b502b1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:49.135818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-85186638-655e-42f8-8bea-7ffa76b502b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:49.229184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 506/996] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:49.511630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e953-1a09-49e0-b81e-ab005495e222 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:49.513245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5840e953-1a09-49e0-b81e-ab005495e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:49.613287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e953-1a09-49e0-b81e-ab005495e222 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:49.685505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:49.686070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:49.686347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:49.686615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:49.688221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:49.694861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:49.793290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:49.794712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ccb9e-10dc-4b64-a3b7-e448b56d4a88 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:49.811950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 491/997] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:33:52.602131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a749a-e679-4f67-a3c1-87e765381e54 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:52.602645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a749a-e679-4f67-a3c1-87e765381e54 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:52.602737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a749a-e679-4f67-a3c1-87e765381e54 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:52.604065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 507/998] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:52.609623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327376d-8d32-4917-9afc-70ff8c22f23a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:52.610047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327376d-8d32-4917-9afc-70ff8c22f23a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:52.610171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327376d-8d32-4917-9afc-70ff8c22f23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:52.682928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6327376d-8d32-4917-9afc-70ff8c22f23a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:52.684195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6327376d-8d32-4917-9afc-70ff8c22f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:52.745679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:52.788088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 492/999] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:53.069824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a749a-e679-4f67-a3c1-87e765381e54 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:53.071481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-016a749a-e679-4f67-a3c1-87e765381e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:53.092910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:53.178015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a749a-e679-4f67-a3c1-87e765381e54 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:53.250137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:53.250749 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:53.251027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:53.251301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:53.253490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:53.325321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:53.363024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:53.364504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecae5e2-fbe3-4e0e-9265-b50575215947 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:53.383121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 508/1000] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:33:55.605595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9396777-fbea-4e24-8de3-c2323f5de6c0 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:55.606136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9396777-fbea-4e24-8de3-c2323f5de6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:55.606136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9396777-fbea-4e24-8de3-c2323f5de6c0 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:55.607518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 493/1001] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:55.613389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7d06-c50d-40c8-b8c5-59d66a80e33b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:55.613811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7d06-c50d-40c8-b8c5-59d66a80e33b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:55.613965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7d06-c50d-40c8-b8c5-59d66a80e33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:55.687729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-63ff7d06-c50d-40c8-b8c5-59d66a80e33b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:55.688946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-63ff7d06-c50d-40c8-b8c5-59d66a80e33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:55.784145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 509/1002] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:56.062760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9396777-fbea-4e24-8de3-c2323f5de6c0 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:56.064330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b9396777-fbea-4e24-8de3-c2323f5de6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:56.168701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9396777-fbea-4e24-8de3-c2323f5de6c0 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:56.243679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:56.244286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:56.244566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:56.244841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:56.246436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:56.263355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:56.354864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:56.356343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713861c-abb1-4e12-8ab3-3b79cd4e9652 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:56.375002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 494/1003] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:33:58.596633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af648c5e-eb56-483d-883c-e8dba1cd0709 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:58.597167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af648c5e-eb56-483d-883c-e8dba1cd0709 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:58.597167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af648c5e-eb56-483d-883c-e8dba1cd0709 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:58.598669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 510/1004] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:33:58.604503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a399f78-795b-4cc2-9a27-712752aa49ae None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:58.604969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a399f78-795b-4cc2-9a27-712752aa49ae None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:58.605063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a399f78-795b-4cc2-9a27-712752aa49ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:58.675383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5a399f78-795b-4cc2-9a27-712752aa49ae None None] MFA Rules not processed for user `291c9818263642f080c993ec53610ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:33:58.676639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5a399f78-795b-4cc2-9a27-712752aa49ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:58.743623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:58.775757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 495/1005] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:33:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:33:59.063740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af648c5e-eb56-483d-883c-e8dba1cd0709 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:33:59.065408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-af648c5e-eb56-483d-883c-e8dba1cd0709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:59.098340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:59.173283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af648c5e-eb56-483d-883c-e8dba1cd0709 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:33:59.245938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:33:59.246543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:33:59.246822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:33:59.247093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:33:59.248709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:33:59.326455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:33:59.360804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:33:59.362196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571bb7e-040a-4a57-9b4e-0a9b6d2b6615 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:33:59.380008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 511/1006] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:33:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:34:01.751594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa29ac-8f4e-4c7f-a4a3-8c008894b80a None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:01.752178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa29ac-8f4e-4c7f-a4a3-8c008894b80a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:01.752178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa29ac-8f4e-4c7f-a4a3-8c008894b80a None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:01.753413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 496/1007] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:01.758521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b2c19-5607-4ac3-ac48-9fb80b738c97 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:01.758966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b2c19-5607-4ac3-ac48-9fb80b738c97 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:01.759078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b2c19-5607-4ac3-ac48-9fb80b738c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:01.822450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9b7b2c19-5607-4ac3-ac48-9fb80b738c97 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:01.823659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9b7b2c19-5607-4ac3-ac48-9fb80b738c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:01.916466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 512/1008] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:02.200652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa29ac-8f4e-4c7f-a4a3-8c008894b80a None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:02.202178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3baa29ac-8f4e-4c7f-a4a3-8c008894b80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:02.303853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa29ac-8f4e-4c7f-a4a3-8c008894b80a None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:02.374672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:02.375271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:02.375586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:02.375859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:02.378733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:02.409563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:02.490450 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:02.491885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fa038-4331-4572-810c-3a473839a1ef service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:02.509870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 497/1009] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:04.679150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb6475-536d-45c4-9593-982510ef6151 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:04.679715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb6475-536d-45c4-9593-982510ef6151 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:04.679715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb6475-536d-45c4-9593-982510ef6151 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:04.681094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 513/1010] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:04.686373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf8971-7ef2-4660-8f34-3f7604f6be86 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:04.686785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf8971-7ef2-4660-8f34-3f7604f6be86 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:04.686930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf8971-7ef2-4660-8f34-3f7604f6be86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:04.752219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-7bbf8971-7ef2-4660-8f34-3f7604f6be86 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:04.753398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7bbf8971-7ef2-4660-8f34-3f7604f6be86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:04.825680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:04.848002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 498/1011] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:05.129451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb6475-536d-45c4-9593-982510ef6151 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:05.130984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d9eb6475-536d-45c4-9593-982510ef6151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:05.170311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:05.231841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb6475-536d-45c4-9593-982510ef6151 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:05.303043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:05.303665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:05.303939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:05.304203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:05.305751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:05.387991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:05.414125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:05.415616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee079d-e5dd-453d-8f40-07f51e956672 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:05.434750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 514/1012] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:07.881347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8e24c-b99b-44f7-a6dd-3d6c99222d36 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:07.881865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8e24c-b99b-44f7-a6dd-3d6c99222d36 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:07.881976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8e24c-b99b-44f7-a6dd-3d6c99222d36 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:07.883262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 499/1013] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:34:07.888521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf416f8-7e9d-4ed6-b3af-cee0d3be95ec None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:07.888964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf416f8-7e9d-4ed6-b3af-cee0d3be95ec None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:07.889183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf416f8-7e9d-4ed6-b3af-cee0d3be95ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:07.954659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-dbf416f8-7e9d-4ed6-b3af-cee0d3be95ec None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:07.955882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dbf416f8-7e9d-4ed6-b3af-cee0d3be95ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:08.050766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 515/1014] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:08.359734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddadd8-030f-4de0-98c1-475f210b6c92 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:08.360004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddadd8-030f-4de0-98c1-475f210b6c92 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:08.360278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddadd8-030f-4de0-98c1-475f210b6c92 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:08.361626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 500/1015] 10.222.0.39 () {54 vars in 964 bytes} [Tue Oct 11 06:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:08.367326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aec952-c257-46d6-b350-1c14b5a38e68 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:08.367641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aec952-c257-46d6-b350-1c14b5a38e68 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:08.367912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aec952-c257-46d6-b350-1c14b5a38e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:08.424713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:08.446154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-35aec952-c257-46d6-b350-1c14b5a38e68 None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:08.447341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-35aec952-c257-46d6-b350-1c14b5a38e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:08.521231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 516/1016] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Oct 11 06:34:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:08.530252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddadd8-030f-4de0-98c1-475f210b6c92 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:08.531801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c4ddadd8-030f-4de0-98c1-475f210b6c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:08.632419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddadd8-030f-4de0-98c1-475f210b6c92 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:08.706269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:08.706865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:08.707142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:08.707429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:08.709013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:08.744500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:08.816421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:08.817764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c716e5-9147-4210-b085-3996aa16c4e5 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:08.835668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 501/1017] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:13.417952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5f36c-2696-49a6-8e16-bc8f026aed2f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:13.418398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5f36c-2696-49a6-8e16-bc8f026aed2f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:13.418523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5f36c-2696-49a6-8e16-bc8f026aed2f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:13.419881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 517/1018] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:13.425733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558dc6b-ec67-42b5-b7e0-7ec68e198393 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:13.426124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558dc6b-ec67-42b5-b7e0-7ec68e198393 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:13.426212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558dc6b-ec67-42b5-b7e0-7ec68e198393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:13.493244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4558dc6b-ec67-42b5-b7e0-7ec68e198393 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:13.494679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4558dc6b-ec67-42b5-b7e0-7ec68e198393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:13.581146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:13.592903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 502/1019] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:13.874261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5f36c-2696-49a6-8e16-bc8f026aed2f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:13.875829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6de5f36c-2696-49a6-8e16-bc8f026aed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:13.978313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5f36c-2696-49a6-8e16-bc8f026aed2f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:14.015244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:14.051833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:14.052434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:14.052710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:14.052979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:14.054561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:14.164017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:14.165377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360beacc-3ce0-405e-afc9-217ef4ea98b1 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:14.183352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 518/1020] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:16.944528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11888d6-3e1c-478b-b252-eca53b9e77b5 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:16.945069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11888d6-3e1c-478b-b252-eca53b9e77b5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:16.945156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11888d6-3e1c-478b-b252-eca53b9e77b5 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:16.946462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 503/1021] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:16.951858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e48811-bf75-478f-8556-206fc3432452 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:16.952275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e48811-bf75-478f-8556-206fc3432452 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:16.952528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e48811-bf75-478f-8556-206fc3432452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:17.017578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-97e48811-bf75-478f-8556-206fc3432452 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:17.018778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-97e48811-bf75-478f-8556-206fc3432452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:17.036632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:17.114383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 519/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:17.401761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11888d6-3e1c-478b-b252-eca53b9e77b5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:17.403308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b11888d6-3e1c-478b-b252-eca53b9e77b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:17.512085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11888d6-3e1c-478b-b252-eca53b9e77b5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:17.586337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:17.586948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:17.587230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:17.587541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:17.589129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:17.632030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:17.698731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:17.700098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4515d-a440-497e-a987-6e38d9575c59 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:17.718910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 504/1023] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:22.086996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478ddbf-6dd3-4e78-8f00-4b374716f543 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:22.087567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478ddbf-6dd3-4e78-8f00-4b374716f543 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:22.087567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478ddbf-6dd3-4e78-8f00-4b374716f543 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:22.088922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 520/1024] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:22.095007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c78ea-b59d-497f-b8d9-1fd3fa17d3f3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:22.095447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c78ea-b59d-497f-b8d9-1fd3fa17d3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:22.095601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c78ea-b59d-497f-b8d9-1fd3fa17d3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:22.164792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-cc8c78ea-b59d-497f-b8d9-1fd3fa17d3f3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:22.166006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cc8c78ea-b59d-497f-b8d9-1fd3fa17d3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:22.264716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:22.270305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 505/1025] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:22.553158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478ddbf-6dd3-4e78-8f00-4b374716f543 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:22.554701 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8478ddbf-6dd3-4e78-8f00-4b374716f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:22.657542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478ddbf-6dd3-4e78-8f00-4b374716f543 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:22.705176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:22.731146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:22.731724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:22.732000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:22.732269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:22.733853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:22.840618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:22.842044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fe751-ea5b-46b7-bdb1-12a63ed6ff4f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:22.859359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 521/1026] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:27.689163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381acc-6883-4538-a40a-cce459ee3056 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:27.689691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381acc-6883-4538-a40a-cce459ee3056 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:27.689691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381acc-6883-4538-a40a-cce459ee3056 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:27.691210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 506/1027] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:27.698657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5dcc-2cc8-4dd1-b4a5-75f93f854644 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:27.699671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5dcc-2cc8-4dd1-b4a5-75f93f854644 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:27.700139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5dcc-2cc8-4dd1-b4a5-75f93f854644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:27.803341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-341f5dcc-2cc8-4dd1-b4a5-75f93f854644 None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:27.804952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-341f5dcc-2cc8-4dd1-b4a5-75f93f854644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:27.847395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:27.989709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 522/1028] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:34:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:32.099370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056045-1217-4d7b-b322-51ce0cc42631 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:32.099978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056045-1217-4d7b-b322-51ce0cc42631 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:32.099978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056045-1217-4d7b-b322-51ce0cc42631 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:32.101444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 507/1029] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:32.106420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978fe21-ba81-432d-a253-f7403b82e52e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:32.107248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978fe21-ba81-432d-a253-f7403b82e52e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:32.107774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978fe21-ba81-432d-a253-f7403b82e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:32.188498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8978fe21-ba81-432d-a253-f7403b82e52e None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:32.189720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8978fe21-ba81-432d-a253-f7403b82e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:32.265805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:32.305663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 523/1030] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:34:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:33.811600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4603-023d-4d4b-9474-69f89260ff83 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:33.812710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4603-023d-4d4b-9474-69f89260ff83 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:33.812710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4603-023d-4d4b-9474-69f89260ff83 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:33.813935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 508/1031] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:33.818995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891d15-ee17-4af8-945a-7421bd1d973d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:33.819451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891d15-ee17-4af8-945a-7421bd1d973d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:33.819575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891d15-ee17-4af8-945a-7421bd1d973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:33.895512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ad891d15-ee17-4af8-945a-7421bd1d973d None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:33.896720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ad891d15-ee17-4af8-945a-7421bd1d973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:34.005273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 524/1032] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:34:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:37.442311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca61620-87cb-4771-a804-7c6c45f34487 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:37.442832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca61620-87cb-4771-a804-7c6c45f34487 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:37.442832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca61620-87cb-4771-a804-7c6c45f34487 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:37.444333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 509/1033] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:37.449834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdbca8-dd72-46d2-870a-0609bae1889f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:37.450366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdbca8-dd72-46d2-870a-0609bae1889f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:37.450366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdbca8-dd72-46d2-870a-0609bae1889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:37.487932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:37.521116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-56fdbca8-dd72-46d2-870a-0609bae1889f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:37.522212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-56fdbca8-dd72-46d2-870a-0609bae1889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:37.622458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 525/1034] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:37.930652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e1742-3d01-4c8c-9064-d9f49065a7a9 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:37.931318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e1742-3d01-4c8c-9064-d9f49065a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:37.931807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e1742-3d01-4c8c-9064-d9f49065a7a9 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:37.934285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 510/1035] 10.222.0.39 () {54 vars in 961 bytes} [Tue Oct 11 06:34:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:37.944896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913fec-6c01-4942-87e8-bf6b0294192a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:37.945598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913fec-6c01-4942-87e8-bf6b0294192a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:37.946298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913fec-6c01-4942-87e8-bf6b0294192a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:38.037992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-08913fec-6c01-4942-87e8-bf6b0294192a None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:38.039182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-08913fec-6c01-4942-87e8-bf6b0294192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:38.090041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:38.163143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 526/1036] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Oct 11 06:34:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:34:38.170590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e1742-3d01-4c8c-9064-d9f49065a7a9 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:38.172643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-075e1742-3d01-4c8c-9064-d9f49065a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:38.323966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e1742-3d01-4c8c-9064-d9f49065a7a9 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:38.399002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:38.453107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:38.454475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:38.454987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:38.455642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:38.457661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:38.569602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:38.571692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c87ac4-8d72-4283-aa1a-ec26a5700929 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:38.592834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 511/1037] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:40.950380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33108eb-86ef-421a-80e9-bb195d9a0164 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:40.950869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33108eb-86ef-421a-80e9-bb195d9a0164 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:40.950869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33108eb-86ef-421a-80e9-bb195d9a0164 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:40.960304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 527/1038] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:40.960611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4133-f755-44b9-9e53-cc2dd791fb4d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:40.960611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4133-f755-44b9-9e53-cc2dd791fb4d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:40.960611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4133-f755-44b9-9e53-cc2dd791fb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:41.026284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4bba4133-f755-44b9-9e53-cc2dd791fb4d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:41.027530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4bba4133-f755-44b9-9e53-cc2dd791fb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:41.066371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:41.130567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 512/1039] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:41.425951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33108eb-86ef-421a-80e9-bb195d9a0164 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:41.427483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b33108eb-86ef-421a-80e9-bb195d9a0164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:41.597060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33108eb-86ef-421a-80e9-bb195d9a0164 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:41.609170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:41.730336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:41.730948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:41.731232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:41.731535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:41.733715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:41.826808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:41.828708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d45243-8b0d-4102-b956-eac7709940ea service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:41.847650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 528/1040] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:44.343893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64e970-2214-4754-84b4-54741f44308e None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:44.344417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64e970-2214-4754-84b4-54741f44308e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:44.344417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64e970-2214-4754-84b4-54741f44308e None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:44.345768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 513/1041] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:44.351038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbb286-8c18-435c-9b1b-d1741ce69515 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:44.351484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbb286-8c18-435c-9b1b-d1741ce69515 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:44.351833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbb286-8c18-435c-9b1b-d1741ce69515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:44.368400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:44.427171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b9cbb286-8c18-435c-9b1b-d1741ce69515 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:44.428429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b9cbb286-8c18-435c-9b1b-d1741ce69515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:44.528666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 529/1042] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:44.827237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a105989-df53-441f-a4e6-592704aee6c1 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:44.827740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a105989-df53-441f-a4e6-592704aee6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:44.828020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a105989-df53-441f-a4e6-592704aee6c1 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:44.829349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 514/1043] 10.222.0.39 () {54 vars in 961 bytes} [Tue Oct 11 06:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:44.834890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90777-aa04-4d40-a1f2-fd7e518f57eb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:44.835250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90777-aa04-4d40-a1f2-fd7e518f57eb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:44.835739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf90777-aa04-4d40-a1f2-fd7e518f57eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:44.913856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ecf90777-aa04-4d40-a1f2-fd7e518f57eb None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:44.915078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ecf90777-aa04-4d40-a1f2-fd7e518f57eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:44.951043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:45.038379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 530/1044] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Oct 11 06:34:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:45.046293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a105989-df53-441f-a4e6-592704aee6c1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:45.048079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0a105989-df53-441f-a4e6-592704aee6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:45.200945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a105989-df53-441f-a4e6-592704aee6c1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:45.222297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:45.323191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:45.323838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:45.324109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:45.324384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:45.325974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:45.435560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:45.437237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97a3db-f614-4953-93ed-010ec6546b16 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:45.457005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 515/1045] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:45.579509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9644d7-d396-4689-9ffe-55e96a447f9d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:45.580082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9644d7-d396-4689-9ffe-55e96a447f9d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:45.580614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9644d7-d396-4689-9ffe-55e96a447f9d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:45.582066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 531/1046] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:45.588255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68c7d-2d13-4931-acf9-614fd5b60fa3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:45.588608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68c7d-2d13-4931-acf9-614fd5b60fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:45.588892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68c7d-2d13-4931-acf9-614fd5b60fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:45.627987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:45.672239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b8c68c7d-2d13-4931-acf9-614fd5b60fa3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:45.673675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b8c68c7d-2d13-4931-acf9-614fd5b60fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:45.787575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 516/1047] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:46.123442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e711-d947-4e04-870e-d57e15f91062 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:46.123442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e711-d947-4e04-870e-d57e15f91062 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:46.123442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e711-d947-4e04-870e-d57e15f91062 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:46.124495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 532/1048] 10.222.0.39 () {54 vars in 963 bytes} [Tue Oct 11 06:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:46.129285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ceca-498c-4104-a257-320a4316b045 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:46.129567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ceca-498c-4104-a257-320a4316b045 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:46.129839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ceca-498c-4104-a257-320a4316b045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:46.212773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6129ceca-498c-4104-a257-320a4316b045 None None] MFA Rules not processed for user `463c2cf049454fa281e99b1a864cdf60`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:46.213974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6129ceca-498c-4104-a257-320a4316b045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:46.262705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:46.295626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 517/1049] 10.222.0.39 () {58 vars in 1060 bytes} [Tue Oct 11 06:34:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:46.303006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1e711-d947-4e04-870e-d57e15f91062 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:46.304539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dcf1e711-d947-4e04-870e-d57e15f91062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:46.424891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1e711-d947-4e04-870e-d57e15f91062 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:46.469960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:46.510465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:46.511220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:46.511615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:46.512007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:46.513954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:46.659569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:46.661589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad9ca5-0fbd-429a-a389-90385d618b30 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:46.688391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 533/1050] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:47.671073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaf69d-c2e9-432f-930b-f980c39435f6 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:47.671675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaf69d-c2e9-432f-930b-f980c39435f6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:47.671675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaf69d-c2e9-432f-930b-f980c39435f6 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:47.673016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 518/1051] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:47.678423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234438de-5aa1-4ad2-bac3-c952338dfe16 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:47.678845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234438de-5aa1-4ad2-bac3-c952338dfe16 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:47.678950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234438de-5aa1-4ad2-bac3-c952338dfe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:47.744582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-234438de-5aa1-4ad2-bac3-c952338dfe16 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:47.745782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-234438de-5aa1-4ad2-bac3-c952338dfe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:47.766619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:47.843224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 534/1052] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:48.138502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaf69d-c2e9-432f-930b-f980c39435f6 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:48.140538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c5eaf69d-c2e9-432f-930b-f980c39435f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:48.309590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaf69d-c2e9-432f-930b-f980c39435f6 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:48.361311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:48.429652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:48.430270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:48.430556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:48.430827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:48.432496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:48.522485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:48.524117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8a99-58c7-4224-a196-2d97bc7e4137 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:48.542819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 519/1053] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:50.761423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159e032-f6dc-4ea0-99f3-a40ece45512a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:50.761957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159e032-f6dc-4ea0-99f3-a40ece45512a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:50.762194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159e032-f6dc-4ea0-99f3-a40ece45512a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:50.763705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 535/1054] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:50.768854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99190f-33dd-44e4-b571-f45d3df4305f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:50.769247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99190f-33dd-44e4-b571-f45d3df4305f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:50.769470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99190f-33dd-44e4-b571-f45d3df4305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:50.836305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-7c99190f-33dd-44e4-b571-f45d3df4305f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:50.837491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7c99190f-33dd-44e4-b571-f45d3df4305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:50.845439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:50.945755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 520/1055] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:51.233277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159e032-f6dc-4ea0-99f3-a40ece45512a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:51.234908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2159e032-f6dc-4ea0-99f3-a40ece45512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:51.387154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159e032-f6dc-4ea0-99f3-a40ece45512a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:51.391314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:51.514958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:51.515681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:51.516058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:51.516427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:51.518109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:51.610822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:51.612605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3a9d5-7880-4c7f-b193-5a58a7759bcd service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:51.631318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 536/1056] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:53.846346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30b0ea-252e-4acc-aa68-5269a89fd1d4 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:53.846875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30b0ea-252e-4acc-aa68-5269a89fd1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:53.846875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30b0ea-252e-4acc-aa68-5269a89fd1d4 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:53.848569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 521/1057] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:53.853765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a9d93-ddc2-42af-937d-cd23e077b6a4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:53.854176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a9d93-ddc2-42af-937d-cd23e077b6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:53.854302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a9d93-ddc2-42af-937d-cd23e077b6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:53.866299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:53.925503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b85a9d93-ddc2-42af-937d-cd23e077b6a4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:53.926683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b85a9d93-ddc2-42af-937d-cd23e077b6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:54.055667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 537/1058] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:34:54.364127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30b0ea-252e-4acc-aa68-5269a89fd1d4 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:54.365672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-af30b0ea-252e-4acc-aa68-5269a89fd1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:54.506122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:54.519238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30b0ea-252e-4acc-aa68-5269a89fd1d4 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:54.639548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:54.640081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:54.640365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:54.640637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:54.642223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:54.733185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:54.734772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8dbe-8748-462f-8280-f59fc90f8941 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:54.744730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:54.754423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 522/1059] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:56.926021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63c4b-af19-4ecf-8d75-d113a6c58df5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:56.926555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63c4b-af19-4ecf-8d75-d113a6c58df5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:56.926649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63c4b-af19-4ecf-8d75-d113a6c58df5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:56.928037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 538/1060] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:56.933665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e3d1-58ec-478d-9b57-8359e773c53f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:56.934120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e3d1-58ec-478d-9b57-8359e773c53f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:56.934410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e3d1-58ec-478d-9b57-8359e773c53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:57.009772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4ee2e3d1-58ec-478d-9b57-8359e773c53f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:34:57.011132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4ee2e3d1-58ec-478d-9b57-8359e773c53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:57.107527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 523/1061] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:34:57.397458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63c4b-af19-4ecf-8d75-d113a6c58df5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:34:57.399677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c0c63c4b-af19-4ecf-8d75-d113a6c58df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:57.498196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:57.563852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63c4b-af19-4ecf-8d75-d113a6c58df5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:34:57.690488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:34:57.691391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:57.691762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:57.692111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:57.694137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:34:57.745893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:34:57.786382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:34:57.788034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4c792-2400-4886-901b-85c8b39fa2ae service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:34:57.806797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 539/1062] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:34:59.976029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776f13-8c4e-40d4-a84c-523a15c36025 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:59.976571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776f13-8c4e-40d4-a84c-523a15c36025 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:59.976571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776f13-8c4e-40d4-a84c-523a15c36025 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:34:59.977931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 524/1063] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:34:59.983287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b1326-c4fc-4119-9b63-c7481cc4b4df None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:34:59.983742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b1326-c4fc-4119-9b63-c7481cc4b4df None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:34:59.983863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b1326-c4fc-4119-9b63-c7481cc4b4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:00.049612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b38b1326-c4fc-4119-9b63-c7481cc4b4df None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:00.050838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b38b1326-c4fc-4119-9b63-c7481cc4b4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:00.147852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 540/1064] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:00.441382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776f13-8c4e-40d4-a84c-523a15c36025 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:00.443027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cd776f13-8c4e-40d4-a84c-523a15c36025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:00.519353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:00.589664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776f13-8c4e-40d4-a84c-523a15c36025 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:00.717634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:00.718245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:00.718555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:00.718822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:00.720543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:00.761127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:00.816943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:00.818534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9cda-40da-452c-a1ea-c4c439dfdf5e service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:00.836354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 525/1065] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:01.881068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143a296-e075-4f57-8bfe-f55ce354766d None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:01.882002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143a296-e075-4f57-8bfe-f55ce354766d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:01.882279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143a296-e075-4f57-8bfe-f55ce354766d None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:01.883680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 541/1066] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:01.889564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082c5f-a78c-4e04-b84e-331443201b3f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:01.889994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082c5f-a78c-4e04-b84e-331443201b3f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:01.890228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082c5f-a78c-4e04-b84e-331443201b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:01.956244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2d082c5f-a78c-4e04-b84e-331443201b3f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:01.957437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2d082c5f-a78c-4e04-b84e-331443201b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:02.052176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 526/1067] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:02.351172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf0d5a-df1e-4c9d-8806-1b47751d52df None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:02.351494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf0d5a-df1e-4c9d-8806-1b47751d52df None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:02.351770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf0d5a-df1e-4c9d-8806-1b47751d52df None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:02.353110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 542/1068] 10.222.0.39 () {54 vars in 962 bytes} [Tue Oct 11 06:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:35:02.358210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0148db-fafd-419f-8155-3b4d4f6682bf None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:02.358567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0148db-fafd-419f-8155-3b4d4f6682bf None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:02.358931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0148db-fafd-419f-8155-3b4d4f6682bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:02.374569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:02.439275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0a0148db-fafd-419f-8155-3b4d4f6682bf None None] MFA Rules not processed for user `3b826760f3d747b888ea373cf5021903`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:02.440530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0a0148db-fafd-419f-8155-3b4d4f6682bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:02.516614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 527/1069] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Oct 11 06:35:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:02.525899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf0d5a-df1e-4c9d-8806-1b47751d52df None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:02.527401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4eaf0d5a-df1e-4c9d-8806-1b47751d52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:02.558339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:02.631086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf0d5a-df1e-4c9d-8806-1b47751d52df None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:02.705490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:02.706093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:02.706366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:02.706633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:02.708236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:02.787436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:02.819089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:02.820534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68672df0-e220-4851-a18e-979871d2b9b8 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:02.839001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 543/1070] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:03.023688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd57d27-a8aa-476b-a2e6-314ce3e81d33 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:03.024275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd57d27-a8aa-476b-a2e6-314ce3e81d33 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:03.024275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd57d27-a8aa-476b-a2e6-314ce3e81d33 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:03.025616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 528/1071] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:03.031753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38da52-1b3d-4d7c-b35f-be4517b93125 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:03.032330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38da52-1b3d-4d7c-b35f-be4517b93125 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:03.032437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38da52-1b3d-4d7c-b35f-be4517b93125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:03.105366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0a38da52-1b3d-4d7c-b35f-be4517b93125 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:03.107123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0a38da52-1b3d-4d7c-b35f-be4517b93125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:03.216451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 544/1072] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:03.504070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd57d27-a8aa-476b-a2e6-314ce3e81d33 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:03.505690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bcd57d27-a8aa-476b-a2e6-314ce3e81d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:03.579454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05040301-f2de-46c0-82f5-d03ab09829a1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:03.582118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05040301-f2de-46c0-82f5-d03ab09829a1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:03.582409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05040301-f2de-46c0-82f5-d03ab09829a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:03.613378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:03.629799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:03.652349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-05040301-f2de-46c0-82f5-d03ab09829a1 None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:03.654139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05040301-f2de-46c0-82f5-d03ab09829a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:03.660280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd57d27-a8aa-476b-a2e6-314ce3e81d33 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:03.738361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 545/1073] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:35:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:03.773491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:03.774242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:03.774344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:03.774612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:03.776275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:03.782452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d32a1-20ad-4a5f-99e1-72b11ebd3542 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:03.782808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d32a1-20ad-4a5f-99e1-72b11ebd3542 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:03.783147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d32a1-20ad-4a5f-99e1-72b11ebd3542 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:03.784626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 546/1074] 10.222.0.39 () {54 vars in 943 bytes} [Tue Oct 11 06:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 06:35:03.790830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e4ae6-df16-44b9-9d46-2f77c0e1500c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:03.791110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e4ae6-df16-44b9-9d46-2f77c0e1500c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:03.791377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e4ae6-df16-44b9-9d46-2f77c0e1500c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:03.849571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:03.869406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:03.870487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d64e4ae6-df16-44b9-9d46-2f77c0e1500c None None] MFA Rules not processed for user `498356fa9eaa41e09da95e0a151105dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:03.871001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:03.871744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d64e4ae6-df16-44b9-9d46-2f77c0e1500c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:03.889564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 529/1075] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:03.937200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:03.987254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 547/1076] 10.222.0.39 () {58 vars in 1040 bytes} [Tue Oct 11 06:35:03 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:35:03.997625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:03.998972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:04.094437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f3527-cb4f-4c5d-9020-286cc05f29f7 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:04.199387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:04.200924 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:04.201540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:04.202056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:04.202543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:04.204422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:04.287343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:04.288977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2acde2-5aca-4347-9780-4a148051cf69 service swift] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:04.297700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 530/1077] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:35:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:35:06.105017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081cf91-c953-4e25-8841-e764dc209e04 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:06.105649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081cf91-c953-4e25-8841-e764dc209e04 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:06.105807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081cf91-c953-4e25-8841-e764dc209e04 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:06.107742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 548/1078] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:06.113114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b3f68-aa0b-47c6-990a-19113680ee24 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:06.113592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b3f68-aa0b-47c6-990a-19113680ee24 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:06.113689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b3f68-aa0b-47c6-990a-19113680ee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:06.184830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-218b3f68-aa0b-47c6-990a-19113680ee24 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:06.186045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-218b3f68-aa0b-47c6-990a-19113680ee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:06.283227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:06.291559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 531/1079] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:06.582393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081cf91-c953-4e25-8841-e764dc209e04 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:06.583998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b081cf91-c953-4e25-8841-e764dc209e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:06.756317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081cf91-c953-4e25-8841-e764dc209e04 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:06.802192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:06.885006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:06.885919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:06.886198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:06.886471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:06.888644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:06.987609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:06.989217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd091473-1228-4fd7-9af6-56c1a090f71a service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:07.008255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 549/1080] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:09.221325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d679db-30ee-413b-8eca-bbb8fa73fb6b None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:09.221902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d679db-30ee-413b-8eca-bbb8fa73fb6b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:09.221902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d679db-30ee-413b-8eca-bbb8fa73fb6b None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:09.229304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad1ff5-1620-4e73-b34f-547cfcd0f69b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:09.229865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad1ff5-1620-4e73-b34f-547cfcd0f69b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:09.229865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad1ff5-1620-4e73-b34f-547cfcd0f69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:09.235311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 532/1081] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:35:09 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:09.304492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:09.313674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f4ad1ff5-1620-4e73-b34f-547cfcd0f69b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:09.314902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f4ad1ff5-1620-4e73-b34f-547cfcd0f69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:09.434245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 550/1082] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:09.719984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d679db-30ee-413b-8eca-bbb8fa73fb6b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:09.721518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a2d679db-30ee-413b-8eca-bbb8fa73fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:09.881486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d679db-30ee-413b-8eca-bbb8fa73fb6b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:09.955310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:10.010466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:10.013494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:10.013777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:10.014050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:10.015734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:10.114183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:10.115796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bcd1b-6ec9-45d8-aee4-3ff6ddf6b06c service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:10.137304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 533/1083] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:12.416932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343876-2bcf-4303-889a-39fb668c7caa None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:12.417442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343876-2bcf-4303-889a-39fb668c7caa None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:12.417532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343876-2bcf-4303-889a-39fb668c7caa None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:12.419000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 551/1084] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:12.424432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237152cc-a4c3-4394-8c88-9e42012afd52 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:12.424933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237152cc-a4c3-4394-8c88-9e42012afd52 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:12.425034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237152cc-a4c3-4394-8c88-9e42012afd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:12.497767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-237152cc-a4c3-4394-8c88-9e42012afd52 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:12.498985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-237152cc-a4c3-4394-8c88-9e42012afd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:12.537645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:12.633725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 534/1085] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:12.922998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52343876-2bcf-4303-889a-39fb668c7caa None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:12.924562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-52343876-2bcf-4303-889a-39fb668c7caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:13.053488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:13.084418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52343876-2bcf-4303-889a-39fb668c7caa None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:13.219575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:13.220229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:13.220509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:13.220780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:13.222399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:13.319114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:13.322088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:13.324296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d7dd0-83f0-4036-9bfd-34d56679f51b service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:13.344772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 552/1086] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:22.632847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=30 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:23.342293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:27.083762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848deec5-5be6-441d-a655-e4aead65915d None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:27.084311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848deec5-5be6-441d-a655-e4aead65915d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:27.084311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848deec5-5be6-441d-a655-e4aead65915d None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:27.085644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 535/1087] 10.222.0.39 () {54 vars in 918 bytes} [Tue Oct 11 06:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:27.091006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857990-b092-4c6b-8d1b-05e6bc0cdcc8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:27.091489 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857990-b092-4c6b-8d1b-05e6bc0cdcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:27.091587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857990-b092-4c6b-8d1b-05e6bc0cdcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:27.166551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-77857990-b092-4c6b-8d1b-05e6bc0cdcc8 None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:27.168290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-77857990-b092-4c6b-8d1b-05e6bc0cdcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:27.271808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 553/1088] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Oct 11 06:35:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:27.286336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e98de-7e48-422c-912e-3cfe0f70338f None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:27.286618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e98de-7e48-422c-912e-3cfe0f70338f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:27.286888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e98de-7e48-422c-912e-3cfe0f70338f None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:27.288260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 536/1089] 10.222.0.39 () {54 vars in 918 bytes} [Tue Oct 11 06:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:27.293387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1abe72-bcf1-405a-ae6a-96cfa80d9fbe None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:27.293668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1abe72-bcf1-405a-ae6a-96cfa80d9fbe None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:27.293936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1abe72-bcf1-405a-ae6a-96cfa80d9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:27.364167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:27.367293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5c1abe72-bcf1-405a-ae6a-96cfa80d9fbe None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:27.369201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5c1abe72-bcf1-405a-ae6a-96cfa80d9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:27.483064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 554/1090] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Oct 11 06:35:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:35:30.044803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ceaa-4937-4eb5-b64a-9b4283b51ed6 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:30.045350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ceaa-4937-4eb5-b64a-9b4283b51ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:30.045350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ceaa-4937-4eb5-b64a-9b4283b51ed6 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:30.046678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 537/1091] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:30.051867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e773d87-33af-45bc-8456-dd0d8f406c38 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:30.052259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e773d87-33af-45bc-8456-dd0d8f406c38 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:30.052422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e773d87-33af-45bc-8456-dd0d8f406c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:30.117238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4e773d87-33af-45bc-8456-dd0d8f406c38 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:30.118439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4e773d87-33af-45bc-8456-dd0d8f406c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:30.174646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:30.216207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 555/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:30.506830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6ceaa-4937-4eb5-b64a-9b4283b51ed6 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:30.508503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c9e6ceaa-4937-4eb5-b64a-9b4283b51ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:30.651663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6ceaa-4937-4eb5-b64a-9b4283b51ed6 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:30.722090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:30.763919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:30.764521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:30.764798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:30.765065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:30.766645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:30.853624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:30.855728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4dd74c-dd45-4906-af6e-e05b2b18605a service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:30.874085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 538/1093] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:31.998298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464ddb-927f-40af-876f-75462071ce1b None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:31.998865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464ddb-927f-40af-876f-75462071ce1b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:31.998865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464ddb-927f-40af-876f-75462071ce1b None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:32.000227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 556/1094] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:32.006246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49a839-c534-4647-b524-c4113b361630 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:32.006681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49a839-c534-4647-b524-c4113b361630 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:32.006806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49a839-c534-4647-b524-c4113b361630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:32.083211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6b49a839-c534-4647-b524-c4113b361630 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:32.084479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6b49a839-c534-4647-b524-c4113b361630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:32.110973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:32.180461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 539/1095] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:32.328481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464ddb-927f-40af-876f-75462071ce1b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:32.329975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25464ddb-927f-40af-876f-75462071ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:32.474068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464ddb-927f-40af-876f-75462071ce1b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:32.513961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:32.585156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:32.585710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:32.585983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:32.586247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:32.587838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:32.675399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:32.677005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591073e-028b-4c06-8bd7-f227b8965539 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:32.694527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 557/1096] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:33.817186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31eec2-a222-413f-8417-e6bb1e069976 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:33.817714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31eec2-a222-413f-8417-e6bb1e069976 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:33.817714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31eec2-a222-413f-8417-e6bb1e069976 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:33.819026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 540/1097] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:33.825741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda30a9-13ed-4d5f-b404-eec7c5828212 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:33.826217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda30a9-13ed-4d5f-b404-eec7c5828212 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:33.826322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda30a9-13ed-4d5f-b404-eec7c5828212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:33.879626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:33.903567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2eda30a9-13ed-4d5f-b404-eec7c5828212 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:33.904773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2eda30a9-13ed-4d5f-b404-eec7c5828212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:33.999264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 558/1098] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:34.151566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31eec2-a222-413f-8417-e6bb1e069976 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:34.153058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7b31eec2-a222-413f-8417-e6bb1e069976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:34.294653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31eec2-a222-413f-8417-e6bb1e069976 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:34.306741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:34.409220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:34.409831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:34.410108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:34.410389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:34.411993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:34.499381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:34.501016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c2f53-4117-4566-a241-6fc8fdf115eb service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:34.518853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 541/1099] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:35.619309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdd0f-384c-443b-ba74-86a70722aeb5 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:35.619902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdd0f-384c-443b-ba74-86a70722aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:35.619902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abdd0f-384c-443b-ba74-86a70722aeb5 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:35.621237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 559/1100] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:35.627310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ec916-76fe-428c-b294-bbfb15296757 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:35.627815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ec916-76fe-428c-b294-bbfb15296757 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:35.627962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ec916-76fe-428c-b294-bbfb15296757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:35.649097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:35.705025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5b9ec916-76fe-428c-b294-bbfb15296757 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:35.706163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5b9ec916-76fe-428c-b294-bbfb15296757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:35.799875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 542/1101] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:35.952965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abdd0f-384c-443b-ba74-86a70722aeb5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:35.954468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-47abdd0f-384c-443b-ba74-86a70722aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:36.061227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:36.097681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abdd0f-384c-443b-ba74-86a70722aeb5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:36.209929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:36.210531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:36.210804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:36.211099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:36.212703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:36.287307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:36.301346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:36.302964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dce1c-0de8-4c5a-bb62-cb2ee3527709 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:36.321272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 560/1102] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:37.420639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eaf37-f826-41ad-8446-3b73a400d381 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:37.421119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eaf37-f826-41ad-8446-3b73a400d381 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:37.421245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eaf37-f826-41ad-8446-3b73a400d381 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:37.422525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 543/1103] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:37.428915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8ecf1-be3b-4885-a212-5c039feb8994 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:37.429358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8ecf1-be3b-4885-a212-5c039feb8994 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:37.429486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8ecf1-be3b-4885-a212-5c039feb8994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:37.504792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1ca8ecf1-be3b-4885-a212-5c039feb8994 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:37.505977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1ca8ecf1-be3b-4885-a212-5c039feb8994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:37.599020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 561/1104] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:35:37.749243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eaf37-f826-41ad-8446-3b73a400d381 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:37.750710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-091eaf37-f826-41ad-8446-3b73a400d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:37.831771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:37.895945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eaf37-f826-41ad-8446-3b73a400d381 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:38.009201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:38.009726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:38.010000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:38.010269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:38.011904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:38.059037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:38.099876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:38.101454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32f29c-6299-4d02-a7db-d05781130287 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:38.118848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 544/1105] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:39.171197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b615e5-d685-43d7-b862-cfcc7230d280 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:39.171788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b615e5-d685-43d7-b862-cfcc7230d280 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:39.171788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b615e5-d685-43d7-b862-cfcc7230d280 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:39.173118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 562/1106] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:39.179367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287a43e-75e7-4d15-9b54-e906685e25aa None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:39.179825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287a43e-75e7-4d15-9b54-e906685e25aa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:39.179928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287a43e-75e7-4d15-9b54-e906685e25aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:39.254006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2287a43e-75e7-4d15-9b54-e906685e25aa None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:39.255184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2287a43e-75e7-4d15-9b54-e906685e25aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:39.347172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 545/1107] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:39.498313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b615e5-d685-43d7-b862-cfcc7230d280 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:39.499865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a2b615e5-d685-43d7-b862-cfcc7230d280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:39.548037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:39.643752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b615e5-d685-43d7-b862-cfcc7230d280 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:39.753063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:39.753663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:39.753939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:39.754207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:39.755823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:39.762882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:39.844445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:39.846029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701699-5ca6-42e4-9d5c-ac03b7a937cb service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:39.864584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 563/1108] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:40.910281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fefad-f2fb-4abb-bbe3-d51e3392bc92 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:40.910861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fefad-f2fb-4abb-bbe3-d51e3392bc92 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:40.910861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fefad-f2fb-4abb-bbe3-d51e3392bc92 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:40.912276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 546/1109] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:40.918513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819851-8658-499f-98cd-6f4b387c0dfd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:40.918968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819851-8658-499f-98cd-6f4b387c0dfd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:40.919062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819851-8658-499f-98cd-6f4b387c0dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:40.993718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ad819851-8658-499f-98cd-6f4b387c0dfd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:40.995429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ad819851-8658-499f-98cd-6f4b387c0dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:41.062486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:41.091905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 564/1110] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:41.245236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fefad-f2fb-4abb-bbe3-d51e3392bc92 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:41.246663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f17fefad-f2fb-4abb-bbe3-d51e3392bc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:41.264233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:41.389338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fefad-f2fb-4abb-bbe3-d51e3392bc92 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:41.475692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:41.501962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:41.502495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:41.502770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:41.503039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:41.504643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:41.591548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:41.593123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec394811-a50f-4eb3-9bf5-6aacb8805877 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:41.610985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 547/1111] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:42.645371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff3d5f-6440-4a30-8fcf-11367593b3f9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:42.645930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff3d5f-6440-4a30-8fcf-11367593b3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:42.645930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff3d5f-6440-4a30-8fcf-11367593b3f9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:42.647248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 565/1112] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:42.653947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ec232-f76a-451a-8974-8b614ab775ef None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:42.654423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ec232-f76a-451a-8974-8b614ab775ef None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:42.654527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ec232-f76a-451a-8974-8b614ab775ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:42.728954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f66ec232-f76a-451a-8974-8b614ab775ef None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:42.730154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f66ec232-f76a-451a-8974-8b614ab775ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:42.770407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:42.826996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 548/1113] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:42.976420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff3d5f-6440-4a30-8fcf-11367593b3f9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:42.977927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-27ff3d5f-6440-4a30-8fcf-11367593b3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:43.121404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff3d5f-6440-4a30-8fcf-11367593b3f9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:43.174871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:43.232470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:43.233006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:43.233279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:43.233545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:43.235115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:43.323771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:43.325357 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c520f-f2e4-4053-9da3-e06090f240f3 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:43.342705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 566/1114] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:44.386365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ebdfa-eb42-4491-a4a3-16f10dfdaed5 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:44.386888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ebdfa-eb42-4491-a4a3-16f10dfdaed5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:44.386888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ebdfa-eb42-4491-a4a3-16f10dfdaed5 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:44.388294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 549/1115] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:44.394251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3793dc-4630-4b4d-99e3-d299d16dcf9c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:44.394983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3793dc-4630-4b4d-99e3-d299d16dcf9c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:44.395092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3793dc-4630-4b4d-99e3-d299d16dcf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:44.470694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8f3793dc-4630-4b4d-99e3-d299d16dcf9c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:44.471934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8f3793dc-4630-4b4d-99e3-d299d16dcf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:44.473923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:44.567213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 567/1116] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:44.717492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ebdfa-eb42-4491-a4a3-16f10dfdaed5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:44.718991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-678ebdfa-eb42-4491-a4a3-16f10dfdaed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:44.863369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ebdfa-eb42-4491-a4a3-16f10dfdaed5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:44.891338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:44.974908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88525988-09db-4c77-b214-980475733dfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:44.975530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88525988-09db-4c77-b214-980475733dfe service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:44.975804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88525988-09db-4c77-b214-980475733dfe service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:44.976069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88525988-09db-4c77-b214-980475733dfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:44.977638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-88525988-09db-4c77-b214-980475733dfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:45.063745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88525988-09db-4c77-b214-980475733dfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:45.065310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88525988-09db-4c77-b214-980475733dfe service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:45.082463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 550/1117] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:47.144502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81897e-7674-41e1-83ca-744d38c73623 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:47.145138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81897e-7674-41e1-83ca-744d38c73623 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:47.145296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81897e-7674-41e1-83ca-744d38c73623 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:47.147111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 568/1118] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:47.153891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa0e6-1e9c-4952-baf0-144203449ac8 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:47.154377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa0e6-1e9c-4952-baf0-144203449ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:47.154486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa0e6-1e9c-4952-baf0-144203449ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:47.194792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:47.220592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4a1fa0e6-1e9c-4952-baf0-144203449ac8 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:47.221779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4a1fa0e6-1e9c-4952-baf0-144203449ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:47.315002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 551/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:47.602420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81897e-7674-41e1-83ca-744d38c73623 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:47.604004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6a81897e-7674-41e1-83ca-744d38c73623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:47.704395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81897e-7674-41e1-83ca-744d38c73623 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:47.735294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:47.777479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:47.778082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:47.778356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:47.778625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:47.780234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:47.889520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:47.890969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc00ac3-cff7-4254-9c0b-f9c482260c40 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:47.909489 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 569/1120] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:50.191108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f599c4-872d-4d9d-8c60-f83d19abfd34 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:50.191679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f599c4-872d-4d9d-8c60-f83d19abfd34 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:50.191679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f599c4-872d-4d9d-8c60-f83d19abfd34 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:50.193043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 552/1121] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:50.198874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632fd6b-d806-4567-8568-9be0c5b18f6f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:50.199299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632fd6b-d806-4567-8568-9be0c5b18f6f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:50.199446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632fd6b-d806-4567-8568-9be0c5b18f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:50.263721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b632fd6b-d806-4567-8568-9be0c5b18f6f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:50.264912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b632fd6b-d806-4567-8568-9be0c5b18f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:50.277523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:50.360054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 570/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:50.642775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f599c4-872d-4d9d-8c60-f83d19abfd34 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:50.644610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c5f599c4-872d-4d9d-8c60-f83d19abfd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:50.746150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f599c4-872d-4d9d-8c60-f83d19abfd34 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:50.758524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:50.821899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:50.823036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:50.823313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:50.823614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:50.825203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:50.931306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:50.933249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b61586-4ce4-4725-9bbc-8032c0279faf service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:50.950581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 553/1123] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:53.399665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e4f9c-3672-47f7-a0bd-6db0ae89c2f3 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:53.400312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e4f9c-3672-47f7-a0bd-6db0ae89c2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:53.400312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e4f9c-3672-47f7-a0bd-6db0ae89c2f3 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:53.401602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 571/1124] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:53.407294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1d6b-5d5c-4b79-923b-961674dc65ca None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:53.407294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1d6b-5d5c-4b79-923b-961674dc65ca None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:53.407805 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1d6b-5d5c-4b79-923b-961674dc65ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:53.454708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:53.472870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-350f1d6b-5d5c-4b79-923b-961674dc65ca None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:53.474071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-350f1d6b-5d5c-4b79-923b-961674dc65ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:53.567913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 554/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:53.854004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e4f9c-3672-47f7-a0bd-6db0ae89c2f3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:53.855599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-378e4f9c-3672-47f7-a0bd-6db0ae89c2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:53.955699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e4f9c-3672-47f7-a0bd-6db0ae89c2f3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:53.994520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:54.029452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:54.030052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:54.030329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:54.030595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:54.032202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:54.137761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:54.139186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22f100-5f27-4731-97ca-6063351048b5 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:54.159938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 572/1126] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:56.399158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191d78-790e-4236-bafc-817003720e29 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:56.399719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191d78-790e-4236-bafc-817003720e29 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:56.399719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191d78-790e-4236-bafc-817003720e29 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:56.401800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 555/1127] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:56.407355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e90c-6742-481a-af62-7386d59d096e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:56.407769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e90c-6742-481a-af62-7386d59d096e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:56.407933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e90c-6742-481a-af62-7386d59d096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:56.472316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4930e90c-6742-481a-af62-7386d59d096e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:56.473506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4930e90c-6742-481a-af62-7386d59d096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:56.493437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:56.570148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 573/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:56.853767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74191d78-790e-4236-bafc-817003720e29 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:56.856549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-74191d78-790e-4236-bafc-817003720e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:56.960173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74191d78-790e-4236-bafc-817003720e29 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:35:56.980402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:57.032539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:35:57.033139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:57.033418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:57.033688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:57.035265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:57.141916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:35:57.143337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d249f-a73f-4797-a343-52bb22ee3171 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:35:57.162297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 556/1129] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:35:59.428386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7343a-02d8-4f08-842b-81e006024e00 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:59.428855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7343a-02d8-4f08-842b-81e006024e00 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:59.428964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7343a-02d8-4f08-842b-81e006024e00 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:59.430284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 574/1130] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:35:59.435847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb1461-3f02-4cd5-adcc-1ada0a243ebc None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:35:59.436304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb1461-3f02-4cd5-adcc-1ada0a243ebc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:35:59.436408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb1461-3f02-4cd5-adcc-1ada0a243ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:35:59.498547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:35:59.501203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8dfb1461-3f02-4cd5-adcc-1ada0a243ebc None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:35:59.503068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8dfb1461-3f02-4cd5-adcc-1ada0a243ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:59.597956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 557/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:35:59.881555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7343a-02d8-4f08-842b-81e006024e00 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:35:59.883100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1e7343a-02d8-4f08-842b-81e006024e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:35:59.983674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7343a-02d8-4f08-842b-81e006024e00 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:00.031656 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:00.056416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:00.057019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:00.057296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:00.057565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:00.059137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:00.164731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:00.166603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebe0b7-2501-4f49-82d9-e1093e6655d6 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:00.184609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 575/1132] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:02.388623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d38b1-77f7-46cd-9bf4-2747e13ff936 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:02.389135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d38b1-77f7-46cd-9bf4-2747e13ff936 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:02.389228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d38b1-77f7-46cd-9bf4-2747e13ff936 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:02.390506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 558/1133] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:02.396143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d400a-2be6-45d9-a7e5-84f5097cb418 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:02.396556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d400a-2be6-45d9-a7e5-84f5097cb418 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:02.396695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d400a-2be6-45d9-a7e5-84f5097cb418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:02.463650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6b2d400a-2be6-45d9-a7e5-84f5097cb418 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:02.464847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6b2d400a-2be6-45d9-a7e5-84f5097cb418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:02.492943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:02.559938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 576/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:02.849437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d38b1-77f7-46cd-9bf4-2747e13ff936 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:02.851004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7c5d38b1-77f7-46cd-9bf4-2747e13ff936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:02.953970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d38b1-77f7-46cd-9bf4-2747e13ff936 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:02.981478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:03.025922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:03.026520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:03.026797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:03.027067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:03.028669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:03.134946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:03.136397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf414e04-9f06-41e1-b8a0-cc6042b49722 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:03.154521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 559/1135] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:05.356053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf324dd-df5d-424a-ad8c-1380e1456c3c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:05.356637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf324dd-df5d-424a-ad8c-1380e1456c3c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:05.356637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf324dd-df5d-424a-ad8c-1380e1456c3c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:05.357950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 577/1136] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:05.363242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d1eee-2571-421f-9c60-507dbbdd4432 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:05.363664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d1eee-2571-421f-9c60-507dbbdd4432 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:05.363851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d1eee-2571-421f-9c60-507dbbdd4432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:05.429211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-009d1eee-2571-421f-9c60-507dbbdd4432 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:05.431060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-009d1eee-2571-421f-9c60-507dbbdd4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:05.439140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:05.527540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 560/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:05.811527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf324dd-df5d-424a-ad8c-1380e1456c3c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:05.813060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cdf324dd-df5d-424a-ad8c-1380e1456c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:05.915795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf324dd-df5d-424a-ad8c-1380e1456c3c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:05.971371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:05.989175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:05.989761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:05.990038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:05.990322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:05.991924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:06.099760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:06.101190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9173-a828-4fe0-9efa-ed2f52b26a3c service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:06.118599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 578/1138] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:08.333367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab34f1-24f4-4d24-8cd7-4f5e8812f096 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:08.333822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab34f1-24f4-4d24-8cd7-4f5e8812f096 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:08.333931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab34f1-24f4-4d24-8cd7-4f5e8812f096 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:08.335245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 561/1139] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:08.340761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e8eb-29d1-48d7-b90c-c6c03e333731 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:08.341198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e8eb-29d1-48d7-b90c-c6c03e333731 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:08.341301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805e8eb-29d1-48d7-b90c-c6c03e333731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:08.408520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a805e8eb-29d1-48d7-b90c-c6c03e333731 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:08.409729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a805e8eb-29d1-48d7-b90c-c6c03e333731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:08.448198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:08.505045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 579/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:08.796394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab34f1-24f4-4d24-8cd7-4f5e8812f096 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:08.797940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b9ab34f1-24f4-4d24-8cd7-4f5e8812f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:08.898584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab34f1-24f4-4d24-8cd7-4f5e8812f096 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:08.936895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:08.973061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:08.973665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:08.973946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:08.974219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:08.975845 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:09.082655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:09.084050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15583726-a85f-4f9e-9c9d-0ccf4ef4baf4 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:09.103712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 562/1141] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:11.240281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c089f5b-5b76-47cd-8639-d4d07fa0458c None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:11.240863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c089f5b-5b76-47cd-8639-d4d07fa0458c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:11.240863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c089f5b-5b76-47cd-8639-d4d07fa0458c None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:11.242186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 580/1142] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:11.248046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2912ba-3252-44d9-9785-0ce63dcde531 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:11.248500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2912ba-3252-44d9-9785-0ce63dcde531 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:11.248596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2912ba-3252-44d9-9785-0ce63dcde531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:11.313399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-df2912ba-3252-44d9-9785-0ce63dcde531 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:11.314595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-df2912ba-3252-44d9-9785-0ce63dcde531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:11.332870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:11.409677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 563/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:11.689123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c089f5b-5b76-47cd-8639-d4d07fa0458c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:11.690718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0c089f5b-5b76-47cd-8639-d4d07fa0458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:11.792770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c089f5b-5b76-47cd-8639-d4d07fa0458c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:11.857864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:11.864675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:11.865305 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:11.865586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:11.865858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:11.867475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:11.974999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:11.976394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d38832-38d2-4761-afb4-7e558c0567d5 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:11.994071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 581/1144] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:14.136383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78d38a-f06c-4811-89b2-70352828b781 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:14.136957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78d38a-f06c-4811-89b2-70352828b781 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:14.136957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78d38a-f06c-4811-89b2-70352828b781 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:14.138271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 564/1145] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:14.143863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6abf9-f74a-4ca1-ab39-9f6b7f782a39 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:14.144292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6abf9-f74a-4ca1-ab39-9f6b7f782a39 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:14.144444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6abf9-f74a-4ca1-ab39-9f6b7f782a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:14.211457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-35d6abf9-f74a-4ca1-ab39-9f6b7f782a39 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:14.212647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-35d6abf9-f74a-4ca1-ab39-9f6b7f782a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:14.261449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:14.308955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 582/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:14.592543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78d38a-f06c-4811-89b2-70352828b781 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:14.594132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8d78d38a-f06c-4811-89b2-70352828b781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:14.694755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78d38a-f06c-4811-89b2-70352828b781 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:14.740905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:14.766852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:14.767478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:14.767758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:14.768284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:14.769863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:14.874987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:14.876443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f618f1-9920-4485-9bdd-179ec259b8a2 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:14.894233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 565/1147] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:36:24.307442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=22 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:24.891745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=66 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:34.686958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29183f-aea3-4ce7-b9c1-2f71f998d37a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:34.687608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29183f-aea3-4ce7-b9c1-2f71f998d37a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:34.687608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29183f-aea3-4ce7-b9c1-2f71f998d37a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:34.688920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 583/1148] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:34.694025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e3f5-454c-4cd7-9fef-d7c97012556f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:34.694421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e3f5-454c-4cd7-9fef-d7c97012556f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:34.694565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e3f5-454c-4cd7-9fef-d7c97012556f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:34.760413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-af73e3f5-454c-4cd7-9fef-d7c97012556f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:34.761600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-af73e3f5-454c-4cd7-9fef-d7c97012556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:34.855988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 566/1149] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:35.153304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432335c0-68de-41ad-8269-69d19c3c64ed None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:35.153591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432335c0-68de-41ad-8269-69d19c3c64ed None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:35.153869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432335c0-68de-41ad-8269-69d19c3c64ed None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:35.155205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 584/1150] 10.222.0.39 () {54 vars in 962 bytes} [Tue Oct 11 06:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:35.160358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ae63-f5fa-4cf6-a741-dd144dae387e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:35.160648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ae63-f5fa-4cf6-a741-dd144dae387e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:35.160924 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ae63-f5fa-4cf6-a741-dd144dae387e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:35.235552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-20a6ae63-f5fa-4cf6-a741-dd144dae387e None None] MFA Rules not processed for user `3b826760f3d747b888ea373cf5021903`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:35.236751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-20a6ae63-f5fa-4cf6-a741-dd144dae387e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:35.256517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:35.311543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 567/1151] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Oct 11 06:36:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:35.319220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432335c0-68de-41ad-8269-69d19c3c64ed None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:35.320799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-432335c0-68de-41ad-8269-69d19c3c64ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:35.423994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432335c0-68de-41ad-8269-69d19c3c64ed None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:35.496528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:35.497131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:35.497410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:35.497680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:35.499259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:35.551850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:35.605837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:35.607296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0b3fe-0dd8-41a4-b482-1fee58976cc4 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:35.625903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 585/1152] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:37.859152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4c34-ec77-48e6-b8ef-fa277b4f7a67 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:37.859776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4c34-ec77-48e6-b8ef-fa277b4f7a67 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:37.859776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4c34-ec77-48e6-b8ef-fa277b4f7a67 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:37.861340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 568/1153] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:37.867204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36518f-6748-43f7-a830-85dfb588d6f3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:37.867700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36518f-6748-43f7-a830-85dfb588d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:37.867811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36518f-6748-43f7-a830-85dfb588d6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:37.935920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9a36518f-6748-43f7-a830-85dfb588d6f3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:37.937123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9a36518f-6748-43f7-a830-85dfb588d6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:38.033369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 586/1154] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:38.310103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4c34-ec77-48e6-b8ef-fa277b4f7a67 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:38.311581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-693c4c34-ec77-48e6-b8ef-fa277b4f7a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:38.434282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4c34-ec77-48e6-b8ef-fa277b4f7a67 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:38.483761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:38.526255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:38.526854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:38.527134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:38.527443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:38.528084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:38.529175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:36:38.529175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:36:38.529175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:36:38.530307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468cd4-23be-458b-af04-04cd032276c2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:38.549143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 569/1155] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Oct 11 06:36:38 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:36:40.726120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ea4da-a646-4158-a2b0-b9dc79c9b6af None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:40.726637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ea4da-a646-4158-a2b0-b9dc79c9b6af None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:40.726732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ea4da-a646-4158-a2b0-b9dc79c9b6af None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:40.728599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 587/1156] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:40.733973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d946bab-66f3-4620-8149-3b4a4c70719c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:40.734397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d946bab-66f3-4620-8149-3b4a4c70719c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:40.734517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d946bab-66f3-4620-8149-3b4a4c70719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:40.799723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8d946bab-66f3-4620-8149-3b4a4c70719c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:40.800912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8d946bab-66f3-4620-8149-3b4a4c70719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:40.893739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 570/1157] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:41.192460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ea4da-a646-4158-a2b0-b9dc79c9b6af None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:41.193970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4c2ea4da-a646-4158-a2b0-b9dc79c9b6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:41.203689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:41.336876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ea4da-a646-4158-a2b0-b9dc79c9b6af None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:41.416416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:41.448649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:41.449187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:41.449466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:41.449736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:41.451323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:41.539409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:41.541099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae6b4e-9fb3-4460-8359-00f291f142c0 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:41.558774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 588/1158] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:43.867739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e4667-e36a-4ae7-afcc-f71ac4b7a2a5 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:43.868371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e4667-e36a-4ae7-afcc-f71ac4b7a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:43.868371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e4667-e36a-4ae7-afcc-f71ac4b7a2a5 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:43.870142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 571/1159] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:43.875341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295647d-c330-4fab-a7f9-d6c1ae56f747 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:43.875799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295647d-c330-4fab-a7f9-d6c1ae56f747 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:43.875898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295647d-c330-4fab-a7f9-d6c1ae56f747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:43.940317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d295647d-c330-4fab-a7f9-d6c1ae56f747 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:43.941504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d295647d-c330-4fab-a7f9-d6c1ae56f747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:43.987395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:44.036915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 589/1160] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:44.327137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e4667-e36a-4ae7-afcc-f71ac4b7a2a5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:44.328693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7b6e4667-e36a-4ae7-afcc-f71ac4b7a2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:44.352128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:44.432895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e4667-e36a-4ae7-afcc-f71ac4b7a2a5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:44.504696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:44.505301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:44.505577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:44.505854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:44.507445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:44.580921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:44.614872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:44.616331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6956f-71ad-416b-8253-cfe0280442e9 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:44.635382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 572/1161] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:46.806150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feb734-d44f-45bd-bd84-e9458246d2c9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:46.806718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feb734-d44f-45bd-bd84-e9458246d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:46.806718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feb734-d44f-45bd-bd84-e9458246d2c9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:46.808089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 590/1162] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:46.813298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582cc81-08fa-4ce2-b817-c26a0dcda469 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:46.813679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582cc81-08fa-4ce2-b817-c26a0dcda469 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:46.813839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582cc81-08fa-4ce2-b817-c26a0dcda469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:46.877928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8582cc81-08fa-4ce2-b817-c26a0dcda469 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:46.879124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8582cc81-08fa-4ce2-b817-c26a0dcda469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:46.972955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 573/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:47.270344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feb734-d44f-45bd-bd84-e9458246d2c9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:47.272152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93feb734-d44f-45bd-bd84-e9458246d2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:47.373854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feb734-d44f-45bd-bd84-e9458246d2c9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:47.445864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:47.447331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:47.447968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:47.448441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:47.448927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:47.450783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:47.557614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:47.558984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334241b-4d7a-441d-bc1c-da6632adf692 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:47.576715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 591/1164] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:36:49.866456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b268dd-f478-4296-b6e5-4146cb809d12 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:49.867017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b268dd-f478-4296-b6e5-4146cb809d12 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:49.867017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b268dd-f478-4296-b6e5-4146cb809d12 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:49.868373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 574/1165] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:49.873644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b6cb9-3b43-448c-acd8-5ebd74032235 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:49.874049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b6cb9-3b43-448c-acd8-5ebd74032235 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:49.874189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b6cb9-3b43-448c-acd8-5ebd74032235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:49.939394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f55b6cb9-3b43-448c-acd8-5ebd74032235 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:49.940616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f55b6cb9-3b43-448c-acd8-5ebd74032235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:49.992166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:50.038482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 592/1166] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:50.332235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b268dd-f478-4296-b6e5-4146cb809d12 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:50.333760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-75b268dd-f478-4296-b6e5-4146cb809d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:50.369788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:50.437451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b268dd-f478-4296-b6e5-4146cb809d12 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:50.509360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:50.510504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:50.510810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:50.511090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:50.512720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:50.592218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:50.621117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:50.622545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6eaf0-bc77-4f28-9384-d0bd417e98c6 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:50.640288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 575/1167] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:36:53.077849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9af5a6-0f38-4bf0-943d-63c351ed9a7a None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:53.078393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9af5a6-0f38-4bf0-943d-63c351ed9a7a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:53.078393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9af5a6-0f38-4bf0-943d-63c351ed9a7a None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:53.079782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 593/1168] 10.222.0.39 () {54 vars in 927 bytes} [Tue Oct 11 06:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:36:53.085333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feca2e6-810c-4a8f-b04a-7b0d11788610 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:53.085750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feca2e6-810c-4a8f-b04a-7b0d11788610 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:53.085865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feca2e6-810c-4a8f-b04a-7b0d11788610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:53.151227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3feca2e6-810c-4a8f-b04a-7b0d11788610 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:36:53.152510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3feca2e6-810c-4a8f-b04a-7b0d11788610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:53.247029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 576/1169] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Oct 11 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:36:53.536198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9af5a6-0f38-4bf0-943d-63c351ed9a7a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:36:53.537719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-db9af5a6-0f38-4bf0-943d-63c351ed9a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:53.642826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9af5a6-0f38-4bf0-943d-63c351ed9a7a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:36:53.714568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:36:53.715093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:36:53.715380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:36:53.715686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:36:53.717274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:36:53.726551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:36:53.824926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:36:53.826373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08066c48-fe6b-45fa-8fd9-0521404d616e service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:36:53.844180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 594/1170] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:37:03.245518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=82 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:37:03.841951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=50 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:12.849844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:12.850402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:12.850402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:12.925004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:38:12.926203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:13.018906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 577/1171] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:38:13.027029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:13.027560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:13.027666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:13.104825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:38:13.105956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:13.206419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 595/1172] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:38:13.210802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:13.212034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:13.224141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfe25e-d491-498d-9ea4-b6507996b324 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:13.286977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:13.323362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:13.323999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:13.324274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:13.324541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:13.325185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:38:13.325483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:13.326488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:38:13.326488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:38:13.326488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:38:13.328102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:13.339510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 578/1173] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:38:13.350527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:13.359965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e886a-f2ae-477c-8199-79836ac56490 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:13.424135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:13.457852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:13.458458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:13.458738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:13.459007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:13.459663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:13.460768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:38:13.460768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:38:13.460768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:38:13.461966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:13.516762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 596/1174] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:38:13 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:38:13.520995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:13.522535 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:13.555827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf46e-c0aa-434f-b14d-5fccc85f1435 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:13.649770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:13.650379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:13.650657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:13.650924 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:13.651830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:13.652828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:38:13.652828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:38:13.652828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:38:13.654053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:13.695646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 579/1175] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:38:13 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:38:13.699654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:13.701165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:13.763002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac827d-1501-42ee-bc41-a86e50d3169a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:13.845353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:13.856087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:13.856692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:13.856967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:13.857235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:13.858454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:13.860091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:13.872200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 597/1176] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:38:13.876193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:13.877715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:13.888808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd9a44-b8ac-493b-b338-058977ad36a5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:13.913628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:13.982009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efe90c-862b-4fc6-b99e-7dff36555fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:13.982612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efe90c-862b-4fc6-b99e-7dff36555fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:13.982892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efe90c-862b-4fc6-b99e-7dff36555fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:13.983789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efe90c-862b-4fc6-b99e-7dff36555fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:13.984435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efe90c-862b-4fc6-b99e-7dff36555fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:13.986016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efe90c-862b-4fc6-b99e-7dff36555fd3 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:13.997959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 580/1177] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:38:14.002140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:14.003700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:14.036729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc592f1a-8abd-4d33-982c-8fd17b98a006 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:14.127726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313a887-3892-430b-97bd-a8b06a19cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:14.128379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a887-3892-430b-97bd-a8b06a19cd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:14.128655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a887-3892-430b-97bd-a8b06a19cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:14.128922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313a887-3892-430b-97bd-a8b06a19cd2f admin admin] PATH_INFO: `/v3/projects/d14ec1847b484285a7e3bd337fe5488e/users/e1623286b9c6464b95dee238c919c0d5/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:14.151365 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313a887-3892-430b-97bd-a8b06a19cd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14ec1847b484285a7e3bd337fe5488e, user_id=e1623286b9c6464b95dee238c919c0d5, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:14.154139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313a887-3892-430b-97bd-a8b06a19cd2f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:14.179887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 598/1178] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:38:14 2022] PUT /identity/v3/projects/d14ec1847b484285a7e3bd337fe5488e/users/e1623286b9c6464b95dee238c919c0d5/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:38:14.191002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dfbcb-ffdf-4807-ad73-3e3278e3d5a9 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:14.191288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dfbcb-ffdf-4807-ad73-3e3278e3d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:14.191608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dfbcb-ffdf-4807-ad73-3e3278e3d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:14.246795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e46dfbcb-ffdf-4807-ad73-3e3278e3d5a9 None None] MFA Rules not processed for user `e1623286b9c6464b95dee238c919c0d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:38:14.248001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e46dfbcb-ffdf-4807-ad73-3e3278e3d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:14.321730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 581/1179] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:38:14.335389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810128-54e9-44c5-a63b-126083e73fa1 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:14.335728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810128-54e9-44c5-a63b-126083e73fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:14.336003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810128-54e9-44c5-a63b-126083e73fa1 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:14.337339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 599/1180] 10.222.0.39 () {54 vars in 963 bytes} [Tue Oct 11 06:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:38:14.342185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04440cd9-d847-457a-9027-db3a1e98ac2c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:14.343054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04440cd9-d847-457a-9027-db3a1e98ac2c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:14.343358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04440cd9-d847-457a-9027-db3a1e98ac2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:14.374200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:14.420429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-04440cd9-d847-457a-9027-db3a1e98ac2c None None] MFA Rules not processed for user `463c2cf049454fa281e99b1a864cdf60`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:38:14.421618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-04440cd9-d847-457a-9027-db3a1e98ac2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:14.493257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 582/1181] 10.222.0.39 () {58 vars in 1060 bytes} [Tue Oct 11 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:38:14.501449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810128-54e9-44c5-a63b-126083e73fa1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:14.502946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-23810128-54e9-44c5-a63b-126083e73fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:14.553625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:14.606317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810128-54e9-44c5-a63b-126083e73fa1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:14.680666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:14.681263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:14.681535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:14.681798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:14.683358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:14.778919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:14.780329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:14.783775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:14.798342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 600/1182] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:38:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:38:14.843721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ddd6-994c-46df-856f-13cc46006bcc None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:14.844006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ddd6-994c-46df-856f-13cc46006bcc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:14.844277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ddd6-994c-46df-856f-13cc46006bcc None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:14.845531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 583/1183] 10.222.0.39 () {52 vars in 830 bytes} [Tue Oct 11 06:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:38:14.849167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:14.850370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:14.948077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb485d-2c40-4543-893e-0fa5ded0ba99 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:15.037021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:15.037632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:15.037835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:15.038099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] PATH_INFO: `/v3/users/e1623286b9c6464b95dee238c919c0d5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:15.050900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1623286b9c6464b95dee238c919c0d5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:15.051945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:38:15.051945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:38:15.051945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:38:15.053141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:15.062942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:15.117116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1623286b9c6464b95dee238c919c0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:15.117640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1623286b9c6464b95dee238c919c0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:15.132320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1623286b9c6464b95dee238c919c0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:15.132760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1623286b9c6464b95dee238c919c0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:15.141684 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1623286b9c6464b95dee238c919c0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:15.169893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 601/1184] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:38:14 2022] DELETE /identity/v3/users/e1623286b9c6464b95dee238c919c0d5 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:38:15.174769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:15.175254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:15.175361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:15.231502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:38:15.232712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:15.307015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:15.319562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 584/1185] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:38:15.329385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:15.330613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:15.435423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10308cbc-f411-4673-87d5-db435172c023 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:15.489976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:15.510786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:15.511393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:15.511704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:15.511975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:15.513553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:15.621048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:15.622497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712562d-414f-4bd8-aed0-71b2ef7c1b99 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:15.641120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 602/1186] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:38:15.983368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:38:15.984660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:38:15.996025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dbe8d-9eff-4c8b-b581-7cffdf8e8d53 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:38:16.095301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:38:16.095962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:38:16.096246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:38:16.096517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] PATH_INFO: `/v3/projects/d14ec1847b484285a7e3bd337fe5488e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:38:16.106212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14ec1847b484285a7e3bd337fe5488e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:38:16.107248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:38:16.107248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:38:16.107248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:38:16.108544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:38:16.166482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14ec1847b484285a7e3bd337fe5488e', 'request_id': 'req-748e41ea-20eb-4479-a79d-55ecbbf053f9'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:16.176792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14ec1847b484285a7e3bd337fe5488e', 'request_id': 'req-748e41ea-20eb-4479-a79d-55ecbbf053f9'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:16.192081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14ec1847b484285a7e3bd337fe5488e', 'request_id': 'req-748e41ea-20eb-4479-a79d-55ecbbf053f9'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:16.214333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] The token cache is being invalidate because project d14ec1847b484285a7e3bd337fe5488e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:38:16.214627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:16.215056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-748e41ea-20eb-4479-a79d-55ecbbf053f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:38:16.216482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 585/1187] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:38:15 2022] DELETE /identity/v3/projects/d14ec1847b484285a7e3bd337fe5488e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:38:25.638392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=62 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:38:26.212302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:39:41.555179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416077a-9159-4757-b5a0-19f3fe1279d6 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:41.555815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416077a-9159-4757-b5a0-19f3fe1279d6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:41.555815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416077a-9159-4757-b5a0-19f3fe1279d6 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:41.557600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 603/1188] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:39:41.562626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3d882-4022-4d7d-a1bc-7e9d8ec3d4f0 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:41.563061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3d882-4022-4d7d-a1bc-7e9d8ec3d4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:41.563161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3d882-4022-4d7d-a1bc-7e9d8ec3d4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:41.641949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-eee3d882-4022-4d7d-a1bc-7e9d8ec3d4f0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:39:41.643147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-eee3d882-4022-4d7d-a1bc-7e9d8ec3d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:41.745979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 586/1189] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:39:41.899097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416077a-9159-4757-b5a0-19f3fe1279d6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:39:41.901238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4416077a-9159-4757-b5a0-19f3fe1279d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:42.018861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416077a-9159-4757-b5a0-19f3fe1279d6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:39:42.024360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a894-2d28-4f9c-9561-0e347b0b787d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:39:42.025480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a894-2d28-4f9c-9561-0e347b0b787d service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:42.025762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a894-2d28-4f9c-9561-0e347b0b787d service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:42.026032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a894-2d28-4f9c-9561-0e347b0b787d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:42.034395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03a894-2d28-4f9c-9561-0e347b0b787d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:39:42.036016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03a894-2d28-4f9c-9561-0e347b0b787d service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:39:42.045838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 604/1190] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:39:42.854268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae2e42-8b4d-410a-b46c-083ad5f49aa4 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:42.854844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae2e42-8b4d-410a-b46c-083ad5f49aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:42.854844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae2e42-8b4d-410a-b46c-083ad5f49aa4 None None] PATH_INFO: `/v3` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:42.856163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 587/1191] 10.222.0.16 () {54 vars in 925 bytes} [Tue Oct 11 06:39:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:39:42.863359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f015d03-0386-40b8-a4a7-40b360be0187 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:42.863838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f015d03-0386-40b8-a4a7-40b360be0187 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:42.863951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f015d03-0386-40b8-a4a7-40b360be0187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:42.923137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3f015d03-0386-40b8-a4a7-40b360be0187 None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:39:42.924371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3f015d03-0386-40b8-a4a7-40b360be0187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:42.990438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:39:42.996349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 605/1192] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Oct 11 06:39:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:39:43.124252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89738c99-b108-46c1-82e5-bbb8de7be88b None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:43.124472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89738c99-b108-46c1-82e5-bbb8de7be88b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:43.124713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89738c99-b108-46c1-82e5-bbb8de7be88b None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:43.126070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 588/1193] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:39:43.132264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d23b3-aff2-4891-9b12-613e1c15e08f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:43.132552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d23b3-aff2-4891-9b12-613e1c15e08f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:43.132828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d23b3-aff2-4891-9b12-613e1c15e08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:43.152153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0e6d23b3-aff2-4891-9b12-613e1c15e08f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:39:43.153342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0e6d23b3-aff2-4891-9b12-613e1c15e08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:43.161278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 606/1194] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:39:43.312773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89738c99-b108-46c1-82e5-bbb8de7be88b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:39:43.326690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89738c99-b108-46c1-82e5-bbb8de7be88b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:39:43.332041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f47e4c-72a9-42ba-bd13-247b224efe54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:39:43.332628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f47e4c-72a9-42ba-bd13-247b224efe54 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:43.332906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f47e4c-72a9-42ba-bd13-247b224efe54 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:43.333175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f47e4c-72a9-42ba-bd13-247b224efe54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:43.341192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f47e4c-72a9-42ba-bd13-247b224efe54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:39:43.342842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f47e4c-72a9-42ba-bd13-247b224efe54 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:39:43.352337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 589/1195] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:39:51.653307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240b25b-773b-453b-9c83-ef74d5a19b56 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:51.653798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240b25b-773b-453b-9c83-ef74d5a19b56 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:51.653910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240b25b-773b-453b-9c83-ef74d5a19b56 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:51.655550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 607/1196] 10.222.0.39 () {54 vars in 961 bytes} [Tue Oct 11 06:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:39:51.660424 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a635-1bfe-4b82-9ee5-c79f01b94362 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:51.660854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a635-1bfe-4b82-9ee5-c79f01b94362 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:51.660969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a635-1bfe-4b82-9ee5-c79f01b94362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:51.680960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6b91a635-1bfe-4b82-9ee5-c79f01b94362 None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:39:51.682773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6b91a635-1bfe-4b82-9ee5-c79f01b94362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:51.691258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 590/1197] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Oct 11 06:39:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:39:51.699149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240b25b-773b-453b-9c83-ef74d5a19b56 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:39:51.712182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240b25b-773b-453b-9c83-ef74d5a19b56 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:39:51.717572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0658ed-2e62-478e-a06b-fd086fe6a7b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:39:51.718153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0658ed-2e62-478e-a06b-fd086fe6a7b8 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:51.718429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0658ed-2e62-478e-a06b-fd086fe6a7b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:51.718698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0658ed-2e62-478e-a06b-fd086fe6a7b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:51.726692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0658ed-2e62-478e-a06b-fd086fe6a7b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:39:51.728301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0658ed-2e62-478e-a06b-fd086fe6a7b8 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:39:51.738362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 608/1198] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:39:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:39:54.819717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040bc6d-dde9-42bc-9403-42bcafb8c939 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:54.820249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040bc6d-dde9-42bc-9403-42bcafb8c939 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:54.820249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040bc6d-dde9-42bc-9403-42bcafb8c939 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:54.821512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 591/1199] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:39:54.827556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1ade1-edf1-4fe5-86c8-96e85672096e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:54.828002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1ade1-edf1-4fe5-86c8-96e85672096e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:54.828107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1ade1-edf1-4fe5-86c8-96e85672096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:54.847929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-66d1ade1-edf1-4fe5-86c8-96e85672096e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:39:54.849172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-66d1ade1-edf1-4fe5-86c8-96e85672096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:54.857657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 609/1200] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:39:55.010309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040bc6d-dde9-42bc-9403-42bcafb8c939 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:39:55.024772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040bc6d-dde9-42bc-9403-42bcafb8c939 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:39:55.030801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cac30-5ead-4619-937a-31c53a00e94a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:39:55.031388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cac30-5ead-4619-937a-31c53a00e94a service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:55.031688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cac30-5ead-4619-937a-31c53a00e94a service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:55.031957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cac30-5ead-4619-937a-31c53a00e94a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:55.039697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cac30-5ead-4619-937a-31c53a00e94a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:39:55.041268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cac30-5ead-4619-937a-31c53a00e94a service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:39:55.050852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 592/1201] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:39:56.120993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59e46-b200-4539-9b1f-e30f25bb8359 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:56.121543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59e46-b200-4539-9b1f-e30f25bb8359 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:56.121543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59e46-b200-4539-9b1f-e30f25bb8359 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:56.122887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 610/1202] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 06:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:39:56.129154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3f24-b20f-4577-b4f1-50f2902f8ae2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:56.129596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3f24-b20f-4577-b4f1-50f2902f8ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:56.129705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3f24-b20f-4577-b4f1-50f2902f8ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:56.149004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fedc3f24-b20f-4577-b4f1-50f2902f8ae2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:39:56.150192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fedc3f24-b20f-4577-b4f1-50f2902f8ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:39:56.158241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 593/1203] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 06:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:39:56.310307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f59e46-b200-4539-9b1f-e30f25bb8359 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:39:56.324492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f59e46-b200-4539-9b1f-e30f25bb8359 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:39:56.329817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6715-d9dd-4695-a4b2-fc29e9994c3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:39:56.330403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6715-d9dd-4695-a4b2-fc29e9994c3c service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:39:56.330682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6715-d9dd-4695-a4b2-fc29e9994c3c service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:39:56.330955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6715-d9dd-4695-a4b2-fc29e9994c3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:39:56.339378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6715-d9dd-4695-a4b2-fc29e9994c3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:39:56.341002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6715-d9dd-4695-a4b2-fc29e9994c3c service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:39:56.350394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 611/1204] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:40:00.258349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3ee7-e031-47ef-8db3-d6033496b2ef None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:00.258913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3ee7-e031-47ef-8db3-d6033496b2ef None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:00.259034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3ee7-e031-47ef-8db3-d6033496b2ef None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:00.260960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 594/1205] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:40:00.266046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27374e1-ae47-435f-8559-2457116ae469 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:00.266483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27374e1-ae47-435f-8559-2457116ae469 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:00.266590 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27374e1-ae47-435f-8559-2457116ae469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:00.285832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f27374e1-ae47-435f-8559-2457116ae469 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:40:00.287023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f27374e1-ae47-435f-8559-2457116ae469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:00.295293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 612/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:40:00.582412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3ee7-e031-47ef-8db3-d6033496b2ef None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:40:00.584162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2fca3ee7-e031-47ef-8db3-d6033496b2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:00.616238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:40:00.661165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3ee7-e031-47ef-8db3-d6033496b2ef None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:40:00.664690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47138b6b-bb13-412c-94fe-982085af8ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:40:00.665317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47138b6b-bb13-412c-94fe-982085af8ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:00.665595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47138b6b-bb13-412c-94fe-982085af8ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:00.665868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47138b6b-bb13-412c-94fe-982085af8ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:00.675620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47138b6b-bb13-412c-94fe-982085af8ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:40:00.677042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47138b6b-bb13-412c-94fe-982085af8ac7 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:40:00.686654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 595/1207] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:40:02.811527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6561-5a7c-4ac3-8d15-7f975f8d44dd None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:02.812050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6561-5a7c-4ac3-8d15-7f975f8d44dd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:02.812050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6561-5a7c-4ac3-8d15-7f975f8d44dd None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:02.813327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 613/1208] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:40:02.818422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ab81f-1482-4606-a928-65c8f193fe77 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:02.818841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ab81f-1482-4606-a928-65c8f193fe77 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:02.818998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ab81f-1482-4606-a928-65c8f193fe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:02.839167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-916ab81f-1482-4606-a928-65c8f193fe77 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:40:02.840404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-916ab81f-1482-4606-a928-65c8f193fe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:02.848075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 596/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:40:03.130353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6561-5a7c-4ac3-8d15-7f975f8d44dd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:40:03.132070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3bb6561-5a7c-4ac3-8d15-7f975f8d44dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:03.145381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6561-5a7c-4ac3-8d15-7f975f8d44dd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:40:03.148874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e272f-6c96-4248-9abf-c2868e9f3be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:40:03.149462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e272f-6c96-4248-9abf-c2868e9f3be0 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:03.149738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e272f-6c96-4248-9abf-c2868e9f3be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:03.150007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e272f-6c96-4248-9abf-c2868e9f3be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:03.158898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e272f-6c96-4248-9abf-c2868e9f3be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:40:03.160354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e272f-6c96-4248-9abf-c2868e9f3be0 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:40:03.169992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 614/1210] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:40:06.785919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005189a-98e8-41cc-8519-ce313c602077 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:06.786452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005189a-98e8-41cc-8519-ce313c602077 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:06.786452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005189a-98e8-41cc-8519-ce313c602077 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:06.787862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 597/1211] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:40:06.793925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96041f2-07ed-46cd-8930-99c87af91b32 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:06.794375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96041f2-07ed-46cd-8930-99c87af91b32 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:06.794480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96041f2-07ed-46cd-8930-99c87af91b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:06.813772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c96041f2-07ed-46cd-8930-99c87af91b32 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:40:06.814987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c96041f2-07ed-46cd-8930-99c87af91b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:06.822772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 615/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:40:07.128763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005189a-98e8-41cc-8519-ce313c602077 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:40:07.130557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3005189a-98e8-41cc-8519-ce313c602077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:07.208898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005189a-98e8-41cc-8519-ce313c602077 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:40:07.212608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c9d00-80ac-49dd-8ada-7c97103efab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:40:07.213792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c9d00-80ac-49dd-8ada-7c97103efab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:07.214070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c9d00-80ac-49dd-8ada-7c97103efab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:07.214341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c9d00-80ac-49dd-8ada-7c97103efab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:07.223153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c9d00-80ac-49dd-8ada-7c97103efab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:40:07.224600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c9d00-80ac-49dd-8ada-7c97103efab2 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:40:07.233917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 598/1213] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:40:09.402373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af77a-f27f-4687-afbc-6d960a9441b9 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:09.402874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af77a-f27f-4687-afbc-6d960a9441b9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:09.403000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af77a-f27f-4687-afbc-6d960a9441b9 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:09.404333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 616/1214] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 06:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:40:09.410213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97319a74-4581-4dc2-ac9f-7f950778670d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:09.410664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97319a74-4581-4dc2-ac9f-7f950778670d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:09.410807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97319a74-4581-4dc2-ac9f-7f950778670d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:09.430973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-97319a74-4581-4dc2-ac9f-7f950778670d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:40:09.432199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-97319a74-4581-4dc2-ac9f-7f950778670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:09.439832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 599/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 06:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:40:09.748995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af77a-f27f-4687-afbc-6d960a9441b9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:40:09.750685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be7af77a-f27f-4687-afbc-6d960a9441b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:40:09.764112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af77a-f27f-4687-afbc-6d960a9441b9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:40:09.767625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca2c4f-ba95-4492-8b8a-bd735da38c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:40:09.768232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2c4f-ba95-4492-8b8a-bd735da38c95 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:40:09.768497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2c4f-ba95-4492-8b8a-bd735da38c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:40:09.768769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca2c4f-ba95-4492-8b8a-bd735da38c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:40:09.777532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca2c4f-ba95-4492-8b8a-bd735da38c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:40:09.778988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca2c4f-ba95-4492-8b8a-bd735da38c95 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:40:09.788773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 617/1216] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:40:19.426192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=63 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:40:19.784611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=37 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:32.120749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.121931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.122573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.150363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.152138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.164718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 600/1217] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.175460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.176049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.176049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.211569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.212761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.225266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 618/1218] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.229862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.246058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980d165-8cb8-4d3a-b56f-90c5122e16aa None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.254991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.255664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.255942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.256212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.256875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:32.257175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.259335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.277106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 601/1219] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:32.285669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.296820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b054d-3f9d-49f5-858f-957959ca4ebf None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.303437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb138395-e93d-4c65-bc8a-fa82e4320494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.304077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb138395-e93d-4c65-bc8a-fa82e4320494 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.304354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb138395-e93d-4c65-bc8a-fa82e4320494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.304626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb138395-e93d-4c65-bc8a-fa82e4320494 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.305819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb138395-e93d-4c65-bc8a-fa82e4320494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.311995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb138395-e93d-4c65-bc8a-fa82e4320494 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.368396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 619/1220] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:32.372595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.394548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36543ed-f9fa-44df-8e38-119bfef34db1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.472493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3201686-a205-4000-9a6b-79225398ad73 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.472875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3201686-a205-4000-9a6b-79225398ad73 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.473059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3201686-a205-4000-9a6b-79225398ad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.495696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.496245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.496470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.496749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.497661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.499717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.507924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b3201686-a205-4000-9a6b-79225398ad73 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.509727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b3201686-a205-4000-9a6b-79225398ad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.523521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 620/1221] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.542972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.543294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.543557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.544756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 602/1222] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:32.549084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.561286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a6ed-2968-4560-a599-7e345d740ec3 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.575950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.577161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.578417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.579124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.579407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.579792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.580457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.582050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.590001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 621/1223] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:41:32.603808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 603/1224] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:32.604599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.607828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.614432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6010b1-eaaf-4aa6-9211-0e85466dbe99 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.620368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8728666-df26-4c38-b2fb-2c11abbb06b3 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.623356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.623943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.624222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.624494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.625151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:32.625452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.626480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:41:32.626480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:41:32.626480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:41:32.627370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.637318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.638266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.638550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.638820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.639502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.641165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.644744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 622/1225] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:32.659874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 604/1226] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:41:32.665054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.665283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.677356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d8692-2404-436a-a5c3-4fd4a3dd813c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.685961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479348-0dc9-4baf-bf4e-c40cac90223d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.692100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.692822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.693110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.693383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.694344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.695365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:41:32.695365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:41:32.695365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:41:32.696729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d1dd7-d581-4f68-8a15-6baca75e30ec admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.705873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb029f-f21e-409f-97f5-6c311c0111d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.706563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb029f-f21e-409f-97f5-6c311c0111d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.706841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb029f-f21e-409f-97f5-6c311c0111d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.707111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb029f-f21e-409f-97f5-6c311c0111d2 admin admin] PATH_INFO: `/v3/projects/211c913f67af4c30899f4a7602ddaa5e/users/4bf1711f968a47179cfb0e6444203f7d/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.741323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 605/1227] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:32.742444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463243-501a-41f3-bf69-25898ee61852 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.743360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463243-501a-41f3-bf69-25898ee61852 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.743657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463243-501a-41f3-bf69-25898ee61852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.750564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb029f-f21e-409f-97f5-6c311c0111d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=211c913f67af4c30899f4a7602ddaa5e, user_id=4bf1711f968a47179cfb0e6444203f7d, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.753348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb029f-f21e-409f-97f5-6c311c0111d2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.779971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-35463243-501a-41f3-bf69-25898ee61852 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.782834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 623/1228] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:32 2022] PUT /identity/v3/projects/211c913f67af4c30899f4a7602ddaa5e/users/4bf1711f968a47179cfb0e6444203f7d/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:32.784127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d51c-fd43-4e57-b156-200395ad64a4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.784404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d51c-fd43-4e57-b156-200395ad64a4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.784668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0d51c-fd43-4e57-b156-200395ad64a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.786611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-35463243-501a-41f3-bf69-25898ee61852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.813977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-28b0d51c-fd43-4e57-b156-200395ad64a4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.815179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-28b0d51c-fd43-4e57-b156-200395ad64a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.865831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 606/1229] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.866694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35463243-501a-41f3-bf69-25898ee61852 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:32.879536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35463243-501a-41f3-bf69-25898ee61852 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:32.884444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231534d6-ba2a-471b-9009-27627e8a34ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:32.884876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 624/1230] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.885810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231534d6-ba2a-471b-9009-27627e8a34ab admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.885810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231534d6-ba2a-471b-9009-27627e8a34ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.885810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231534d6-ba2a-471b-9009-27627e8a34ab admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.886131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e285371-4770-438b-9cbc-419d7cf2d7a0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.886131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e285371-4770-438b-9cbc-419d7cf2d7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.887442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e285371-4770-438b-9cbc-419d7cf2d7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.887653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231534d6-ba2a-471b-9009-27627e8a34ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:32.888948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231534d6-ba2a-471b-9009-27627e8a34ab admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:32.908960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5e285371-4770-438b-9cbc-419d7cf2d7a0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.910857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5e285371-4770-438b-9cbc-419d7cf2d7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.919372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 607/1231] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/users => generated 284 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:32.919372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7234784-21bc-44dc-bd60-f054ab22d1c1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.919372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7234784-21bc-44dc-bd60-f054ab22d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.919372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7234784-21bc-44dc-bd60-f054ab22d1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.926566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 625/1232] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.928311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.928800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.929292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.938036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d7234784-21bc-44dc-bd60-f054ab22d1c1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.939257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d7234784-21bc-44dc-bd60-f054ab22d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.952393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 608/1233] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.952393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e27663-ff81-474f-bc55-6d1afd99a121 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.952393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e27663-ff81-474f-bc55-6d1afd99a121 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.952393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e27663-ff81-474f-bc55-6d1afd99a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:32.957898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:32.965400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] MFA Rules not processed for user `4bf1711f968a47179cfb0e6444203f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.967159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.971387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-29e27663-ff81-474f-bc55-6d1afd99a121 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:32.972620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-29e27663-ff81-474f-bc55-6d1afd99a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:32.981284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 609/1234] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:32.987559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bded6c3-d758-4356-8af0-2b7f43832ce5 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:32.987559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bded6c3-d758-4356-8af0-2b7f43832ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:32.987559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bded6c3-d758-4356-8af0-2b7f43832ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.004728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6bded6c3-d758-4356-8af0-2b7f43832ce5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.005948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6bded6c3-d758-4356-8af0-2b7f43832ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.015206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 610/1235] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.016409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a640e9-0328-4bba-b3cc-bee277bc89f2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.016860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a640e9-0328-4bba-b3cc-bee277bc89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.017178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a640e9-0328-4bba-b3cc-bee277bc89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.031596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:33.041060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-52a640e9-0328-4bba-b3cc-bee277bc89f2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.042315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 626/1236] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.042473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-52a640e9-0328-4bba-b3cc-bee277bc89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.043580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.054341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 611/1237] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.055673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f8b14-3c75-4e67-8836-28d6760454f0 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.055959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f8b14-3c75-4e67-8836-28d6760454f0 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.056230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f8b14-3c75-4e67-8836-28d6760454f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.061045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe16c18-78cb-455f-af7e-1dde94d2abcc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.067133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250341-a2e3-4260-a3c0-356c766fc490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.067869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250341-a2e3-4260-a3c0-356c766fc490 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.068217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250341-a2e3-4260-a3c0-356c766fc490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.068555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250341-a2e3-4260-a3c0-356c766fc490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.069339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250341-a2e3-4260-a3c0-356c766fc490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.071158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250341-a2e3-4260-a3c0-356c766fc490 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.076395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9f6f8b14-3c75-4e67-8836-28d6760454f0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.078055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9f6f8b14-3c75-4e67-8836-28d6760454f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.082129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 627/1238] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:33.083515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aecd1d-85df-4b15-986a-f66fad1c3652 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.083875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aecd1d-85df-4b15-986a-f66fad1c3652 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.084223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aecd1d-85df-4b15-986a-f66fad1c3652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.091065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 612/1239] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.092543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a24f0-9897-4254-852d-49ea872326ab None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.092825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a24f0-9897-4254-852d-49ea872326ab None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.093099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a24f0-9897-4254-852d-49ea872326ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.109681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a7aecd1d-85df-4b15-986a-f66fad1c3652 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.110873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a7aecd1d-85df-4b15-986a-f66fad1c3652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.113585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-922a24f0-9897-4254-852d-49ea872326ab None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.114767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-922a24f0-9897-4254-852d-49ea872326ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.118869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 628/1240] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.119882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.120160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.120429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.126616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 613/1241] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.131077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a24f0-9897-4254-852d-49ea872326ab None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.139679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.140630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a24f0-9897-4254-852d-49ea872326ab None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.140872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.145230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.145825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.146101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.146389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.147031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:33.147359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.149550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98c85d-2fd6-4763-ac83-d8dcee3cf14d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.149992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 629/1242] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.150835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.161052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6592d-f42f-48de-a9ba-04729da80f7c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.169259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 614/1243] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:33.169259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ff7dd-3a6f-4531-a1e3-f8d9a451aade None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.169259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ff7dd-3a6f-4531-a1e3-f8d9a451aade None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.169259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ff7dd-3a6f-4531-a1e3-f8d9a451aade None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.169259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 615/1244] 10.222.0.39 () {54 vars in 961 bytes} [Tue Oct 11 06:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:41:33.169790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.169790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.169790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.169790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.170133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:33.170133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.170133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.170434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ff7dd-3a6f-4531-a1e3-f8d9a451aade None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.183016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 630/1245] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:33.183904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.185309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ff7dd-3a6f-4531-a1e3-f8d9a451aade None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.190085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40430b10-7c1b-4dfe-9217-02b4c9818b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.190516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40430b10-7c1b-4dfe-9217-02b4c9818b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.190795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40430b10-7c1b-4dfe-9217-02b4c9818b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.191063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40430b10-7c1b-4dfe-9217-02b4c9818b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.191742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40430b10-7c1b-4dfe-9217-02b4c9818b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.193325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40430b10-7c1b-4dfe-9217-02b4c9818b55 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.194298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529228f-7f84-4075-88c4-29c53ada51f2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.198875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.199514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.199824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.200096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.200757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:33.201055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.203324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.203831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 616/1246] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:33.207318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.207318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.207318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.223690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 631/1247] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:33.224584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.231024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.232277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.235454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959f342-54ae-4828-b035-fdf7c44bc13a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.239942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.240223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 617/1248] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.243854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.243854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.243854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.244134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.244225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:33.244225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.244225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.253418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b2826-5bdc-4923-af46-3c3d00a80199 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.255470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 632/1249] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:33.256254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.258340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.259105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.259385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.259683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.260466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:33.260771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.262642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211bddd-02c3-4e23-8d7d-e0410dba130c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.267510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6fa62-3094-4872-9a01-c730a1d496e8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.272174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.272762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.273040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.273312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.274462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.274873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 618/1250] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:33.276107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.276386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.276680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.280592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.308485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 633/1251] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.312520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.314252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.315511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.376402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58b693-bd52-4264-a821-f94d1aa7d1a8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.380897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.381568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.389571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 619/1252] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.389728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.389728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.389910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.390012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.390012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.398674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc3824-70fe-4576-b424-1da94746b45c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.404195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.405287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.405567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.406063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] PATH_INFO: `/v3/projects/f80702350d5e4a26babbea591f9ae8e2/users/9447a2b3ff604607b746bff05b70a355/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.417985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 634/1253] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.418944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.425233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80702350d5e4a26babbea591f9ae8e2, user_id=9447a2b3ff604607b746bff05b70a355, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.428016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.440836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 620/1254] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:33 2022] PUT /identity/v3/projects/f80702350d5e4a26babbea591f9ae8e2/users/9447a2b3ff604607b746bff05b70a355/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:33.441635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.482915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:33.486645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:33.492199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebb9aa-eea9-4fd7-b0fb-0eb936e2145e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.503593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c44cfe-8bc4-4d55-994e-9de8c017c0b4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.510496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.511162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.511496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.512113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.512417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:33.512720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.514372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.526610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 621/1255] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:33.526610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.537456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d681d3-865f-4d5b-a40e-9240e37a1b08 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.543056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.543675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.544264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.544545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.545201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.547110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.558695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.559302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.559613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.559887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.560516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.562794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.575712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 622/1256] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.576538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.591970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 635/1257] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.593035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.647917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764dcd20-50a5-4af6-b93a-524aaf76f097 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.660455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cf25d-a018-4080-8c2b-269a73086f5b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.665224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.665843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.666123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.666395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.667030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.668113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:41:33.668113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:41:33.668113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:41:33.669324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.692761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 636/1258] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.694070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.705978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.706591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.706874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.707146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.707814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.709714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.739519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 623/1259] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/projects => generated 393 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.739519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.767652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c35f0-9df3-47a9-a436-a4203865f05c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.779655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:33.784937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:33.803207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf633d9e-808b-4666-93ae-be91c9e18bd5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.808482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705a3b7-677d-4ffe-bd1a-4faa912f2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.809090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705a3b7-677d-4ffe-bd1a-4faa912f2725 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.809366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705a3b7-677d-4ffe-bd1a-4faa912f2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.809637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705a3b7-677d-4ffe-bd1a-4faa912f2725 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.810267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705a3b7-677d-4ffe-bd1a-4faa912f2725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.812272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705a3b7-677d-4ffe-bd1a-4faa912f2725 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.834882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 624/1260] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.835123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.835903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.836800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.836912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.837075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.837193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.837385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.864976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] MFA Rules not processed for user `9447a2b3ff604607b746bff05b70a355`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:33.866186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:33.893626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.895545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.905832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 637/1261] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:33.906538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.917016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eba45-2ab8-407f-a125-03ed6b192049 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.921686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.922267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.922543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.922813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.923478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.925383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.934365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 625/1262] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:33.935254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.946748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602a6d5-f853-4376-bfd6-8216d647b99b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:33.953925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:33.954793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:33.955088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:33.955563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:33.956431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:33.961062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 638/1263] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.961062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:33.961840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:33.985967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 626/1264] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:33.987086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.001315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77930ba0-5b2d-4348-b494-e98725ba1ccc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.027908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5d81d-2000-47fb-b8d8-09c09a7934ff admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.032915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.033523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.033800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.034075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.034846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.036796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.059219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 639/1265] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:33 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:34.063021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.069266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.069983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.070266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.070550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.071778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.073395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.073518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471a601-64ff-48db-ae8b-dbf3342178d8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.078170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.078755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.079034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.079306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.080046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.081987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.085329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 627/1266] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.089704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.102312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf0043-4d45-49ee-ab7e-ae3f957debf9 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.109233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 640/1267] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:34 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:34.109233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.110210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.110834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.111247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.111543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.112326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:34.113337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.116726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.133980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 628/1268] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.133980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.142091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9ca51-c949-4c58-942b-d5a6e6b3cba3 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.147617 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.148260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.148535 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.149508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.150163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.152392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.174192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28e5c-214d-4551-a8de-5619afe2306e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.175156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 629/1269] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:34 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:34.176170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.181016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.181623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.181934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.182204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.186106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e860-b426-487a-afe4-24cf3adfcf1b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.190976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.191625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.191908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.192182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.192825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.194413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.205022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 630/1270] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.208930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.214611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:34.220438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932ad6a-74a9-4b2b-b760-db69f8bf85ad admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.227104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.227756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.228059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.228349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.229015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.230589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.241990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 631/1271] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.242513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.243067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.244005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.252989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 641/1272] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:34.253786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.254882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b0a3-2ff1-408c-9875-2d9f09bc1797 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.260264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.260865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.261151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.261478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.262117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.263817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.264439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ff69c-da11-4011-a527-06895f97f996 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.270250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55daa32-2609-4e47-87b2-a814be739e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.270883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55daa32-2609-4e47-87b2-a814be739e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.271166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55daa32-2609-4e47-87b2-a814be739e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.271753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55daa32-2609-4e47-87b2-a814be739e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.272404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55daa32-2609-4e47-87b2-a814be739e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.274557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55daa32-2609-4e47-87b2-a814be739e46 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.279010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 632/1273] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.279010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.286161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d08a-7ed0-4eca-a141-25b0ca68b142 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.288657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 642/1274] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.289939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01900cff-23ff-4f20-8612-ecd8d92a6992 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.290226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01900cff-23ff-4f20-8612-ecd8d92a6992 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.290503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01900cff-23ff-4f20-8612-ecd8d92a6992 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.291388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.291990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.292168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 643/1275] 10.222.0.39 () {54 vars in 912 bytes} [Tue Oct 11 06:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:41:34.292270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.292523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.293044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01900cff-23ff-4f20-8612-ecd8d92a6992 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.293173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.294774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.305047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 633/1276] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.305864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.306740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01900cff-23ff-4f20-8612-ecd8d92a6992 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.312927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.313823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.314112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.314385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.315029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.315629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28dfb2-5e0f-45ae-99da-04c0fca94689 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.316649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.320132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.321261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.321539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.321809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.322439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.324054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.328313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 644/1277] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.332324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.341650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 634/1278] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.341650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.347224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344557-a544-43fa-acc5-1f17d6fb0ea6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.348700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe328ca-bf86-4a07-b75c-ccfd91d44ffe admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.352459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.353065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.353320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821df26-44d8-4d9e-9f70-518005af6cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.353470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.353624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.353965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821df26-44d8-4d9e-9f70-518005af6cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.354250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821df26-44d8-4d9e-9f70-518005af6cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.354399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.354506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821df26-44d8-4d9e-9f70-518005af6cea admin admin] PATH_INFO: `/v3/projects/bf094dc7d5f1453ba9112d1a1d3f0d9c/users/1c602e0f9e0e4e479a6a3c32febdb14b/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.356045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.371443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 645/1279] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.371443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.371712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821df26-44d8-4d9e-9f70-518005af6cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf094dc7d5f1453ba9112d1a1d3f0d9c, user_id=1c602e0f9e0e4e479a6a3c32febdb14b, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.373731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821df26-44d8-4d9e-9f70-518005af6cea admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.381454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affcb9b-2041-4112-bab8-4b48fa3a9b56 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.389564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 635/1280] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:34 2022] PUT /identity/v3/projects/bf094dc7d5f1453ba9112d1a1d3f0d9c/users/1c602e0f9e0e4e479a6a3c32febdb14b/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:34.392673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.393230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.393383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.417993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:34.419251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:34.454294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.454897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.455172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.455457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.456261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.457973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.469312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 646/1281] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.472879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.482950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1c295-7a28-451f-981d-ceee0520f83d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.487643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.488396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.488682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.488956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] PATH_INFO: `/v3/projects/0601aee062c04a51b85224c57d6a433a/users/a3153d5b95464304a6f16677c69fd591/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.494409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 636/1282] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:41:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:34.494409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.506424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0601aee062c04a51b85224c57d6a433a, user_id=a3153d5b95464304a6f16677c69fd591, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.510216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.513397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:34.515922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113acce-e58f-4aaf-9833-de6f15d4e0e8 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.521613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.522249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.522524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.523365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.523570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.525231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.525869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 647/1283] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:34 2022] PUT /identity/v3/projects/0601aee062c04a51b85224c57d6a433a/users/a3153d5b95464304a6f16677c69fd591/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:34.528401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.536929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 637/1284] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:34.537745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.550271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba56a6-7abe-4d78-854a-26cb781da441 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.608893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6171c-1fb2-4be2-92bf-d8609cc3085b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.614125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4fe68-bcbb-4f5e-9f9e-595d873a9f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.614969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fe68-bcbb-4f5e-9f9e-595d873a9f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.615325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fe68-bcbb-4f5e-9f9e-595d873a9f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.616364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fe68-bcbb-4f5e-9f9e-595d873a9f51 admin admin] PATH_INFO: `/v3/projects/5af47e2321de4d299ff3a7fb3d3d9e3c/users/2e63c3f83f3d41018e6330f26ed7b8a0/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.622343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.623109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.623493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.623841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] PATH_INFO: `/v3/projects/ec4a4c53c1c744688b3bf91e806286bf/users/f95c1cd09e1042abbb5725f241ce147d/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.633729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fe68-bcbb-4f5e-9f9e-595d873a9f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af47e2321de4d299ff3a7fb3d3d9e3c, user_id=2e63c3f83f3d41018e6330f26ed7b8a0, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.636890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fe68-bcbb-4f5e-9f9e-595d873a9f51 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.639083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4a4c53c1c744688b3bf91e806286bf, user_id=f95c1cd09e1042abbb5725f241ce147d, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.642367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.650703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:34.652102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 648/1285] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:34 2022] PUT /identity/v3/projects/5af47e2321de4d299ff3a7fb3d3d9e3c/users/2e63c3f83f3d41018e6330f26ed7b8a0/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:34.653226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.653535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.653772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.660106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 638/1286] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:34 2022] PUT /identity/v3/projects/ec4a4c53c1c744688b3bf91e806286bf/users/f95c1cd09e1042abbb5725f241ce147d/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:34.661874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.678781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:34.680789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:34.732712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39b3d3-053a-42bc-93d9-c5de1303e323 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.743860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d68029-b624-4348-96c5-9053b7dde49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.744916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d68029-b624-4348-96c5-9053b7dde49e admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.745364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d68029-b624-4348-96c5-9053b7dde49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.745643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d68029-b624-4348-96c5-9053b7dde49e admin admin] PATH_INFO: `/v3/projects/0e4e977dc2e84b08a4fbaf7fd2fac42f/users/3d57230baeb24e8a8a511a0c6417077e/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.750462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 649/1287] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:34.750462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.768379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d68029-b624-4348-96c5-9053b7dde49e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4e977dc2e84b08a4fbaf7fd2fac42f, user_id=3d57230baeb24e8a8a511a0c6417077e, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.771626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d68029-b624-4348-96c5-9053b7dde49e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.804324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 639/1288] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:34 2022] PUT /identity/v3/projects/0e4e977dc2e84b08a4fbaf7fd2fac42f/users/3d57230baeb24e8a8a511a0c6417077e/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:34.806242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.806636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.806979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.828310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c226e8-9758-48ad-8b31-d6a4f62c1dc2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:34.845597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] MFA Rules not processed for user `a3153d5b95464304a6f16677c69fd591`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:34.846912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:34.873490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:34.894089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce91521-dac0-48d3-945e-a59ed52fb62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:34.894828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91521-dac0-48d3-945e-a59ed52fb62b service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.895292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91521-dac0-48d3-945e-a59ed52fb62b service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.895615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91521-dac0-48d3-945e-a59ed52fb62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:34.939892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 640/1289] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:34.940714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:34.967813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91521-dac0-48d3-945e-a59ed52fb62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:34.969408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91521-dac0-48d3-945e-a59ed52fb62b service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:34.981103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 650/1290] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:34.984102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef669ee9-d5d8-405c-9d66-1ee7229ccd0e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:34.984385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef669ee9-d5d8-405c-9d66-1ee7229ccd0e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:34.984660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef669ee9-d5d8-405c-9d66-1ee7229ccd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.006779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ef669ee9-d5d8-405c-9d66-1ee7229ccd0e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:35.008679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ef669ee9-d5d8-405c-9d66-1ee7229ccd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:35.009497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77f82d-d977-4700-adc2-08cfb762022e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.014105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.014890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.015150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.015437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] PATH_INFO: `/v3/projects/048f1b5e89c8417695e7916b444e2778/users/e97edc31dd834fd8811ad1bb25f1168b/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.017746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 651/1291] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:35.019855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101c06-9f35-470d-af82-9dde2281f260 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.020210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101c06-9f35-470d-af82-9dde2281f260 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.020670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff101c06-9f35-470d-af82-9dde2281f260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.032279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048f1b5e89c8417695e7916b444e2778, user_id=e97edc31dd834fd8811ad1bb25f1168b, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.035168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.038113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:35.042861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ff101c06-9f35-470d-af82-9dde2281f260 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:35.044426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ff101c06-9f35-470d-af82-9dde2281f260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:35.052208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 641/1292] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:34 2022] PUT /identity/v3/projects/048f1b5e89c8417695e7916b444e2778/users/e97edc31dd834fd8811ad1bb25f1168b/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:35.053030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.110224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 652/1293] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:35.111615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.111805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.112080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.118367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44202f66-f961-4897-b83e-df5737a0cec2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.122157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.122808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.123108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.123380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.132372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:35.133441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.133662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:35.135118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.141764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 653/1294] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:35.142648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.146950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 642/1295] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:35.147954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.154280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71764c-9f23-4ce5-a4dc-746a5d160601 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.159000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092f535-d90d-467b-b23b-70756d6df2fa service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.159784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.160452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.160779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.161053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.161740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.163986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.164885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a18a6-58dd-464c-b1b1-919fa5fdf4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.165961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a18a6-58dd-464c-b1b1-919fa5fdf4bf service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.166418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a18a6-58dd-464c-b1b1-919fa5fdf4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.167051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a18a6-58dd-464c-b1b1-919fa5fdf4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.184517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a18a6-58dd-464c-b1b1-919fa5fdf4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.191576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a18a6-58dd-464c-b1b1-919fa5fdf4bf service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.205841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 643/1296] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:35.205984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 654/1297] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:35 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:35.209261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.209429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.209429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.209429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.229190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:35.230830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:35.290145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:35.317077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd0812-5e2e-406e-9c6f-687db62e2bee admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.323461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.324286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.324568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.324858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.329393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 644/1298] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:35.329393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.338407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ff08d-a0d2-44c7-8df7-9b66903815fe None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.341028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.343605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.343834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.344891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.345621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.346142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.361485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.363840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 655/1299] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:35.363980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.364481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.376283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 645/1300] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:35.377167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.389091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c53ca-9237-44f2-9a87-777187b60f7a service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.394230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.395046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.395922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.396198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.397230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8498-5ff1-4fda-9286-1bdc01fbdaf2 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.407956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.408813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.409040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.409293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.410108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.412269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.419518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.421296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.437753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 646/1301] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:35.455863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 656/1302] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:35 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:35.459955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.474196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7a9f6-1096-4a59-b096-44c32a7d1481 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.494613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.497803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.498883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.499410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.500947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.503570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.525059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 647/1303] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:35.526285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.545150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36fdf-076b-4985-a526-566e7ed1fb17 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.558374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.559583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.560171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.560739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] PATH_INFO: `/v3/projects/39eaea8e2f7e445aacdc2bd92cb476bb/users/cc36ce4c3b634db88be251055748386e/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.580900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eaea8e2f7e445aacdc2bd92cb476bb, user_id=cc36ce4c3b634db88be251055748386e, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.583743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.601580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 657/1304] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:35 2022] PUT /identity/v3/projects/39eaea8e2f7e445aacdc2bd92cb476bb/users/cc36ce4c3b634db88be251055748386e/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:35.606965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.633640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5aedf-6fc6-4084-ba02-9b9795712e0d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.733450 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:35.778494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07b03d-45bf-412d-a67c-5bbaaafb6eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.781774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07b03d-45bf-412d-a67c-5bbaaafb6eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.782089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07b03d-45bf-412d-a67c-5bbaaafb6eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.782367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07b03d-45bf-412d-a67c-5bbaaafb6eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.784201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07b03d-45bf-412d-a67c-5bbaaafb6eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.802236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07b03d-45bf-412d-a67c-5bbaaafb6eb0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.867042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:35.880286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 648/1305] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:35.881337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cfcf1-af68-4937-9159-3ea36ebef87c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:35.897403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:35.898094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.898372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.898646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] PATH_INFO: `/v3/projects/39eaea8e2f7e445aacdc2bd92cb476bb/users/cc36ce4c3b634db88be251055748386e/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:35.907164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eaea8e2f7e445aacdc2bd92cb476bb, user_id=cc36ce4c3b634db88be251055748386e, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:35.912354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:35.939836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 658/1306] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:35 2022] PUT /identity/v3/projects/39eaea8e2f7e445aacdc2bd92cb476bb/users/cc36ce4c3b634db88be251055748386e/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:41:35.952144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05fcf-2a3e-469b-a5df-e9bb63caae1c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:35.952639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05fcf-2a3e-469b-a5df-e9bb63caae1c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:35.952923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05fcf-2a3e-469b-a5df-e9bb63caae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.056569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3cc05fcf-2a3e-469b-a5df-e9bb63caae1c None None] MFA Rules not processed for user `cc36ce4c3b634db88be251055748386e`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:36.076311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3cc05fcf-2a3e-469b-a5df-e9bb63caae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:36.266973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 649/1307] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:36.291108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:36.370357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d78d07-2b2e-4335-ad33-2dcdc0b08876 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:36.371184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d78d07-2b2e-4335-ad33-2dcdc0b08876 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:36.372078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d78d07-2b2e-4335-ad33-2dcdc0b08876 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.373879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 650/1308] 10.222.0.39 () {54 vars in 961 bytes} [Tue Oct 11 06:41:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:41:36.382132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49003d8a-8a1b-4373-99e7-4dbbdc7b5cb8 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:36.382419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49003d8a-8a1b-4373-99e7-4dbbdc7b5cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:36.382915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49003d8a-8a1b-4373-99e7-4dbbdc7b5cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.416976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-49003d8a-8a1b-4373-99e7-4dbbdc7b5cb8 None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:36.418198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-49003d8a-8a1b-4373-99e7-4dbbdc7b5cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:36.425126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed83b45-17fc-4a60-bd2e-7463c1fdc68a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:36.429847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57156e36-ddb1-4a1c-9277-1cc2d64dbaa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:36.431218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156e36-ddb1-4a1c-9277-1cc2d64dbaa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:36.431634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156e36-ddb1-4a1c-9277-1cc2d64dbaa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:36.432110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156e36-ddb1-4a1c-9277-1cc2d64dbaa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.444319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57156e36-ddb1-4a1c-9277-1cc2d64dbaa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:36.446317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57156e36-ddb1-4a1c-9277-1cc2d64dbaa0 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:36.459857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 659/1309] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:41:36.565606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 651/1310] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Oct 11 06:41:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:41:36.574527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade234-7e6a-46f6-b7fe-fc06a4571da2 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:36.574872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade234-7e6a-46f6-b7fe-fc06a4571da2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:36.575211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade234-7e6a-46f6-b7fe-fc06a4571da2 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.577466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 660/1311] 10.222.0.39 () {54 vars in 912 bytes} [Tue Oct 11 06:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:41:36.586506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:36.586853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:36.587134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.612369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:36.613689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:36.625485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 652/1312] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:41:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:36.648025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade234-7e6a-46f6-b7fe-fc06a4571da2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:36.672854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade234-7e6a-46f6-b7fe-fc06a4571da2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:36.686873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83f47a-5d3f-42b4-a9d5-6596adfb88f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:36.687662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83f47a-5d3f-42b4-a9d5-6596adfb88f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:36.688911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83f47a-5d3f-42b4-a9d5-6596adfb88f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:36.689285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83f47a-5d3f-42b4-a9d5-6596adfb88f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:36.718479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83f47a-5d3f-42b4-a9d5-6596adfb88f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:36.723553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83f47a-5d3f-42b4-a9d5-6596adfb88f2 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:36.751596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 661/1313] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:37.513338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:37.626831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:37.644945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e45844-c310-40fd-9d6e-3c93051484e1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:37.653543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:37.654619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:37.655234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:37.656198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] PATH_INFO: `/v3/users/a3153d5b95464304a6f16677c69fd591` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:37.659107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3153d5b95464304a6f16677c69fd591)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:37.660262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: WARNING py.warnings [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:41:37.660262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: warnings.warn(msg) Oct 11 06:41:37.660262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: Oct 11 06:41:37.662500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:37.773212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3153d5b95464304a6f16677c69fd591'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:37.775132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3153d5b95464304a6f16677c69fd591'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:37.777086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3153d5b95464304a6f16677c69fd591'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:37.812315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3153d5b95464304a6f16677c69fd591'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:37.835991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3153d5b95464304a6f16677c69fd591'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:37.872955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 653/1314] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:37 2022] DELETE /identity/v3/users/a3153d5b95464304a6f16677c69fd591 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:37.874592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:37.875052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:37.875134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:37.913947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:37.915381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:38.057154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 662/1315] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:38.122616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:38.124897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:38.267247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304e713-1b2f-47db-b1b6-810841e4aad7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:38.273511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453fcb6-e30d-46c6-95a4-61cc8020c1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:38.274414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fcb6-e30d-46c6-95a4-61cc8020c1bb service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:38.274414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fcb6-e30d-46c6-95a4-61cc8020c1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:38.274758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fcb6-e30d-46c6-95a4-61cc8020c1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:38.301050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453fcb6-e30d-46c6-95a4-61cc8020c1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:38.302642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453fcb6-e30d-46c6-95a4-61cc8020c1bb service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:38.333204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 654/1316] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:38.336066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:38.336066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:38.359452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ed84-2eb5-4c2e-a23c-89f1a181959a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:38.377997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:38.378662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:38.378941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:38.379216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] PATH_INFO: `/v3/users/4bf1711f968a47179cfb0e6444203f7d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:38.380411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf1711f968a47179cfb0e6444203f7d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:38.382351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:38.417833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:38.495953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf1711f968a47179cfb0e6444203f7d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:38.497060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf1711f968a47179cfb0e6444203f7d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:38.519058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf1711f968a47179cfb0e6444203f7d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:38.523725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf1711f968a47179cfb0e6444203f7d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:38.557717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf1711f968a47179cfb0e6444203f7d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:38.595019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 663/1317] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:38 2022] DELETE /identity/v3/users/4bf1711f968a47179cfb0e6444203f7d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:38.599531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:38.600071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:38.600071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:38.664013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:38.664013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:38.778203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 655/1318] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:38.798300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:38.801026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:38.890521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447525c9-1b52-4d1d-a6e0-0fba75ed4143 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:38.896576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:38.897455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:38.897873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:38.898261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:38.915009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:38.918411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:38.937116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 664/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:39.533992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:39.535362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:39.548053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff2490-b592-4c69-97c5-ef1fb89daf07 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:39.553233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:39.554275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:39.554275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:39.554471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] PATH_INFO: `/v3/projects/0601aee062c04a51b85224c57d6a433a` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:39.555739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0601aee062c04a51b85224c57d6a433a)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:39.557841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:39.574624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:39.635830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0601aee062c04a51b85224c57d6a433a', 'request_id': 'req-528d0748-c8d5-40b0-a4da-392da98192cc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:39.653616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0601aee062c04a51b85224c57d6a433a', 'request_id': 'req-528d0748-c8d5-40b0-a4da-392da98192cc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:39.662859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0601aee062c04a51b85224c57d6a433a', 'request_id': 'req-528d0748-c8d5-40b0-a4da-392da98192cc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:39.721184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] The token cache is being invalidate because project 0601aee062c04a51b85224c57d6a433a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:39.721509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:39.723203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:39.724810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 656/1320] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:39 2022] DELETE /identity/v3/projects/0601aee062c04a51b85224c57d6a433a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:39.729015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:39.738734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66398e9-e95a-443e-bf2d-81778689b3c1 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:39.778323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:39.782295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:39.888044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:39.889180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:39.890445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:39.890445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] PATH_INFO: `/v3/users/cc36ce4c3b634db88be251055748386e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:39.891608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc36ce4c3b634db88be251055748386e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:39.893180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d0748-c8d5-40b0-a4da-392da98192cc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:39.894923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:39.899949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:39.900689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:39.901043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:39.901426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] PATH_INFO: `/v3/projects/211c913f67af4c30899f4a7602ddaa5e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:39.903133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=211c913f67af4c30899f4a7602ddaa5e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:39.906225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:39.967029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:39.988340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc36ce4c3b634db88be251055748386e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:39.989174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc36ce4c3b634db88be251055748386e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.021841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc36ce4c3b634db88be251055748386e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.022678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc36ce4c3b634db88be251055748386e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.029250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '211c913f67af4c30899f4a7602ddaa5e', 'request_id': 'req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.042256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fc8a86af-154d-4bc0-90fb-74396dfb720a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc36ce4c3b634db88be251055748386e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.042901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '211c913f67af4c30899f4a7602ddaa5e', 'request_id': 'req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.052835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '211c913f67af4c30899f4a7602ddaa5e', 'request_id': 'req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.066009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] The token cache is being invalidate because project 211c913f67af4c30899f4a7602ddaa5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:40.066545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.067355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ef10f78-9a35-4eb3-9ac1-bfb8ce79d485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:40.069380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 657/1321] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:39 2022] DELETE /identity/v3/projects/211c913f67af4c30899f4a7602ddaa5e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:41:40.074381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.075343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.075858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.076605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 665/1322] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:39 2022] DELETE /identity/v3/users/cc36ce4c3b634db88be251055748386e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:40.141887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:40.145178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:40.200338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:40.281731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 658/1323] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:40.283389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.283700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.284276 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.325579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:40.326831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:40.352264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 666/1324] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:40.356411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:40.367521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0341-2b94-425b-8d65-f0da6212ea0b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:40.386240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:40.386891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.387170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.387464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.388143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:40.388448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:40.390719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:40.425340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 659/1325] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:40.429524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:40.441458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf275ae-22dc-41a2-b7ba-c41f8211fc24 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:40.455196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:40.455858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.456326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.456752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.457709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:40.464709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:40.567790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 667/1326] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:40 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:40.572330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:40.594046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ac86f-af0f-44dc-980b-4b567cbeb9b8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:40.665523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:40.667480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:40.757990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:40.759967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.760259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.760676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.761319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:40.763527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:40.812307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 660/1327] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:40 2022] POST /identity/v3/users => generated 282 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:40.812884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:40.823099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a596-0e7a-4280-b001-e4ef28fb8a37 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:40.835298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d798e-8b6a-4b17-bcf3-c8d9b88613ae admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:40.837623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:40.838307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.838660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.838901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] PATH_INFO: `/v3/projects/39eaea8e2f7e445aacdc2bd92cb476bb` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.842159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eaea8e2f7e445aacdc2bd92cb476bb)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:40.842159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: WARNING py.warnings [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 06:41:40.842159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: warnings.warn(msg) Oct 11 06:41:40.842159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: Oct 11 06:41:40.842719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:40.948020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:40.948020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.948020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.948020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.949224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:40.949224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:40.949224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 661/1328] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:41:40.949224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:40.959958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1457bae-56c6-4ebd-9b47-c04cd31e9887 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:40.973265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:40.974724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:40.975493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:40.975778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:40.978862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:40.982441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:41.031780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 662/1329] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:41.035901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:41.057020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e5cb-1e3d-4a4b-a611-16ed22e8ff3f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:41.057869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eaea8e2f7e445aacdc2bd92cb476bb', 'request_id': 'req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:41.111110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eaea8e2f7e445aacdc2bd92cb476bb', 'request_id': 'req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:41.132621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eaea8e2f7e445aacdc2bd92cb476bb', 'request_id': 'req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:41.159565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] The token cache is being invalidate because project 39eaea8e2f7e445aacdc2bd92cb476bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:41.159861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:41.161791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51f6f4a7-7115-4c45-ac71-4e23ecc6cca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:41.169729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 668/1330] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:40 2022] DELETE /identity/v3/projects/39eaea8e2f7e445aacdc2bd92cb476bb => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:41:41.174348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173b5f0-d2bb-4ba0-88af-f776898a19d0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.174675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173b5f0-d2bb-4ba0-88af-f776898a19d0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.174955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173b5f0-d2bb-4ba0-88af-f776898a19d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.239646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6173b5f0-d2bb-4ba0-88af-f776898a19d0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:41.241206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6173b5f0-d2bb-4ba0-88af-f776898a19d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:41.258566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:41.261183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 669/1331] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:41:41.262501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.263248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.263248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] PATH_INFO: `/v3/projects/3b7f470b9c694c7fa3eb596f2c81e9db/users/6157adafab554fff8fd6dda95882499c/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.270497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4fd2e-d05f-46f9-b157-cf51aecee966 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.271072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4fd2e-d05f-46f9-b157-cf51aecee966 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.271544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4fd2e-d05f-46f9-b157-cf51aecee966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.316566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7f470b9c694c7fa3eb596f2c81e9db, user_id=6157adafab554fff8fd6dda95882499c, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:41.324320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:41.327217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d8a4fd2e-d05f-46f9-b157-cf51aecee966 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:41.329168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d8a4fd2e-d05f-46f9-b157-cf51aecee966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:41.354006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 663/1332] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:41 2022] PUT /identity/v3/projects/3b7f470b9c694c7fa3eb596f2c81e9db/users/6157adafab554fff8fd6dda95882499c/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:41.354262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 670/1333] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:41.359083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:41.366564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.369150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.369644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.388020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:41.390414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:41.455634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] MFA Rules not processed for user `6157adafab554fff8fd6dda95882499c`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:41.457567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:41.532138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa726b89-3f07-4094-a4b4-c20a2ef89ced admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:41.544702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:41.545349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.545628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.545902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.548608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:41.548951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:41.550871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:41.592735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 664/1334] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:41.612550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:41.629697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f507058-d286-4edc-8e07-cde28e410caa admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:41.634561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:41.635356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.636624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.636968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.637564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:41.642938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:41.680443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 671/1335] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:41.682422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:41.692430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:41.719317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 665/1336] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:41 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:41.750672 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab6bd-668a-4dc5-9640-2e433711b322 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:41.751960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:41.919459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481d743-7759-43ea-8a7d-47462172db11 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:41.931457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:41.937121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:41.937875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.938161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.938478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.939149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:41.941486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:41.944264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:41.944754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:41.945692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:41.947241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:41.949419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:41.995465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 666/1337] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:41 2022] POST /identity/v3/users => generated 284 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:42.000055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.030349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 672/1338] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:41 2022] POST /identity/v3/projects => generated 365 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:42.031308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.042798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1ab80-7415-4913-aa37-056944c8ac60 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:42.210159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:42.210799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.211626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.211906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.212623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:42.214212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:42.240789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c9027-5708-422a-8e3a-f53823f32231 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:42.254759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 667/1339] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:42.258747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.266904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:42.267863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.268293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.268735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.269555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:42.271882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:42.274172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:42.296024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48551-6687-47db-be57-6cd882e5943d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:42.305232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:42.305878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.306153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.306433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.316123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:42.323518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:42.345757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 673/1340] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:42 2022] POST /identity/v3/users => generated 289 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:42.358974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.367949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 668/1341] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:42.368933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.390874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1179ff-3cc6-41db-b3d4-9a1d9475257f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:42.399704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2af5a-8df0-4834-8c32-84957a1e1ade admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:42.415962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:42.417053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.417558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.424849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.425881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:42.429577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:42.442612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bde5d9-f6f3-4eae-af37-06143fec8f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:42.442612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bde5d9-f6f3-4eae-af37-06143fec8f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.442971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bde5d9-f6f3-4eae-af37-06143fec8f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.443068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bde5d9-f6f3-4eae-af37-06143fec8f69 admin admin] PATH_INFO: `/v3/projects/cb58f4783188412787989f13a9929332/users/02ec4b80e0d94a47ab53e3052bfe57ab/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.451945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:42.474968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 674/1342] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:42.485967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.510226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bde5d9-f6f3-4eae-af37-06143fec8f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb58f4783188412787989f13a9929332, user_id=02ec4b80e0d94a47ab53e3052bfe57ab, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:42.516943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bde5d9-f6f3-4eae-af37-06143fec8f69 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:42.536972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1273e2b-5129-45d4-8e88-21c24c0a3869 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:42.557310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 669/1343] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:42 2022] PUT /identity/v3/projects/cb58f4783188412787989f13a9929332/users/02ec4b80e0d94a47ab53e3052bfe57ab/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:42.568949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.568949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.568949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.577343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:42.578060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:42.578342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:42.583532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] PATH_INFO: `/v3/projects/ba9d9648fcc2477bb3eb12b88ccf5c37/users/f945ccb1091c4f3fb53edb2dbc3445c7/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:42.658245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9d9648fcc2477bb3eb12b88ccf5c37, user_id=f945ccb1091c4f3fb53edb2dbc3445c7, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:42.666379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:42.669045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] MFA Rules not processed for user `02ec4b80e0d94a47ab53e3052bfe57ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:42.671047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:42.700817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 675/1344] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:42 2022] PUT /identity/v3/projects/ba9d9648fcc2477bb3eb12b88ccf5c37/users/f945ccb1091c4f3fb53edb2dbc3445c7/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:42.706304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:42.989679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 670/1345] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:43.002226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.042850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a6d6a-33d6-4ea0-b3e8-fbe5b0823972 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.064670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.068002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.070359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.071530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.075264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:43.084285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:43.138068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 676/1346] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:43.148047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.186162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cb507-f17c-463c-8d01-61210e2dbe1e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.200900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18c177-b069-4c73-aa93-1697c51449fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.201740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c177-b069-4c73-aa93-1697c51449fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.202123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c177-b069-4c73-aa93-1697c51449fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.202646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c177-b069-4c73-aa93-1697c51449fc admin admin] PATH_INFO: `/v3/projects/ba9d9648fcc2477bb3eb12b88ccf5c37/users/f945ccb1091c4f3fb53edb2dbc3445c7/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.210066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18c177-b069-4c73-aa93-1697c51449fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9d9648fcc2477bb3eb12b88ccf5c37, user_id=f945ccb1091c4f3fb53edb2dbc3445c7, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:43.219984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18c177-b069-4c73-aa93-1697c51449fc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:43.260304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 677/1347] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:43 2022] PUT /identity/v3/projects/ba9d9648fcc2477bb3eb12b88ccf5c37/users/f945ccb1091c4f3fb53edb2dbc3445c7/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:43.260810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d2f78-0cb3-417a-a922-ef8d0d886199 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.260910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.260910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.260910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.368348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.369304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.369858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.370361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.371248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:43.371755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] MFA Rules not processed for user `f945ccb1091c4f3fb53edb2dbc3445c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:43.373282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:43.374130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:43.396613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:43.451056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 671/1348] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:43 2022] POST /identity/v3/projects => generated 369 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:41:43.452224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.545352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 678/1349] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:41:43.557317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.560593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:43.563583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40cb3-0b65-452a-8561-da90e2107531 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.576767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.577437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.578208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.578208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.578673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:43.585309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:43.638438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 672/1350] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:43 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:43.640334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.659595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8692-6db8-4894-bf3f-f0a203f4a876 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.662769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:43.677586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.677960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.678251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.678534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.681160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:43.684799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:43.717364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 673/1351] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:43.719642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.725574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2c700-1842-419f-9479-dc8034f60543 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.752233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13c61d-8ba7-46aa-a172-654709d88237 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:43.759213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.759930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.760211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.760483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] PATH_INFO: `/v3/projects/e2581ff2ce4143998d30b233bff330df/users/5c1008b183b3451ea9cea7ce6700849e/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:43.810274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2581ff2ce4143998d30b233bff330df, user_id=5c1008b183b3451ea9cea7ce6700849e, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:43.813329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:43.843647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 674/1352] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:43 2022] PUT /identity/v3/projects/e2581ff2ce4143998d30b233bff330df/users/5c1008b183b3451ea9cea7ce6700849e/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:41:43.848751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:43.896629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:43.897269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:43.897551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:43.897824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:44.020860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb173-1d19-4ae7-83fe-fa9649be3688 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:44.026512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:44.027154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:44.027507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:44.027749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:44.028566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:44.031053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:44.043861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 675/1353] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:44.048218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:44.063042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711c6b-4fe5-4cfd-ab59-8e79a394c27e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:44.070186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc0f86-5f74-4565-a4b9-c40b82f366ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:44.071639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0f86-5f74-4565-a4b9-c40b82f366ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:44.072064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0f86-5f74-4565-a4b9-c40b82f366ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:44.072455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0f86-5f74-4565-a4b9-c40b82f366ff admin admin] PATH_INFO: `/v3/projects/e2581ff2ce4143998d30b233bff330df/users/5c1008b183b3451ea9cea7ce6700849e/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:44.074742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc0f86-5f74-4565-a4b9-c40b82f366ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2581ff2ce4143998d30b233bff330df, user_id=5c1008b183b3451ea9cea7ce6700849e, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:44.077888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc0f86-5f74-4565-a4b9-c40b82f366ff admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:44.102902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 676/1354] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:44 2022] PUT /identity/v3/projects/e2581ff2ce4143998d30b233bff330df/users/5c1008b183b3451ea9cea7ce6700849e/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:44.105853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:44.106455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:44.107105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:44.132247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:44.134262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:44.178618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 679/1355] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:44.188451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] MFA Rules not processed for user `5c1008b183b3451ea9cea7ce6700849e`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:44.191025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:44.246203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:44.388365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 677/1356] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:44.420020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:44.434345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa14f84-c2e4-4fcd-b049-7d826fd6aa17 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:44.590330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:44.592375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:44.592658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:44.594956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:44.633565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:44.635164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:44.662169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 680/1357] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:44.758293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:44.775463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43cae9-fbe4-4125-a58b-f9bcee7184ba None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:44.787437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3702b3-c146-4d65-9287-0a0a8856f8bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:44.788961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3702b3-c146-4d65-9287-0a0a8856f8bc service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:44.791343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3702b3-c146-4d65-9287-0a0a8856f8bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:44.792108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3702b3-c146-4d65-9287-0a0a8856f8bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:44.904362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3702b3-c146-4d65-9287-0a0a8856f8bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:44.906701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3702b3-c146-4d65-9287-0a0a8856f8bc service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:44.939653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 678/1358] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:45.125164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:45.248923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:45.277896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b84df-afa4-47ca-8506-3442738b25af service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:45.290014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:45.291169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:45.291726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:45.292219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] PATH_INFO: `/v3/users/6157adafab554fff8fd6dda95882499c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:45.299433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6157adafab554fff8fd6dda95882499c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:45.305548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:45.443562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6157adafab554fff8fd6dda95882499c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:45.445798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6157adafab554fff8fd6dda95882499c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:45.466345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6157adafab554fff8fd6dda95882499c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:45.469415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6157adafab554fff8fd6dda95882499c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:45.495013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6157adafab554fff8fd6dda95882499c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:45.548414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 681/1359] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:45 2022] DELETE /identity/v3/users/6157adafab554fff8fd6dda95882499c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:45.555640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:45.556227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:45.556227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:45.619382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:45.623574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:45.682776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:45.684333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:45.799121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 679/1360] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:45.816102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:45.817550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:45.876743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8cf33-70d7-4902-ae4a-7796b5c5bbc1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:45.890094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:45.893060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:45.893343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:45.894360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] PATH_INFO: `/v3/users/9447a2b3ff604607b746bff05b70a355` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:45.897668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9447a2b3ff604607b746bff05b70a355)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:45.899794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:45.949646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b997da-19db-4b75-b2d7-a5481da374cf None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:45.954142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de92b8-2782-4041-a1da-50654a310256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:45.955133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de92b8-2782-4041-a1da-50654a310256 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:45.955672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de92b8-2782-4041-a1da-50654a310256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:45.956182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de92b8-2782-4041-a1da-50654a310256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:45.965201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:45.968852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de92b8-2782-4041-a1da-50654a310256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:45.973080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de92b8-2782-4041-a1da-50654a310256 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:45.988680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9447a2b3ff604607b746bff05b70a355'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:45.989981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9447a2b3ff604607b746bff05b70a355'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:46.000115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 680/1361] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:46.011127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9447a2b3ff604607b746bff05b70a355'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:46.012642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9447a2b3ff604607b746bff05b70a355'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:46.025799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9447a2b3ff604607b746bff05b70a355'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:46.036205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:46.055648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 682/1362] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:45 2022] DELETE /identity/v3/users/9447a2b3ff604607b746bff05b70a355 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:46.060427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a463fd-75a2-499f-af19-679479262767 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:46.060870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a463fd-75a2-499f-af19-679479262767 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:46.061324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a463fd-75a2-499f-af19-679479262767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:46.088358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f2a463fd-75a2-499f-af19-679479262767 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:46.090222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f2a463fd-75a2-499f-af19-679479262767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:46.177330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 681/1363] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:41:46.204617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:46.206138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:46.285028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22509c5-365d-47fd-8c0d-7997e2b5d417 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:46.294674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:46.297073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:46.298192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:46.298313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:46.323737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:46.325242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:46.357190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 683/1364] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:46.759405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a463fd-75a2-499f-af19-679479262767 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:46.760833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f2a463fd-75a2-499f-af19-679479262767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:46.784230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a463fd-75a2-499f-af19-679479262767 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:46.797855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:46.799196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:46.800877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:46.801838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] PATH_INFO: `/v3/projects/3b7f470b9c694c7fa3eb596f2c81e9db` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:46.805301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7f470b9c694c7fa3eb596f2c81e9db)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:46.811776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:46.948098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7f470b9c694c7fa3eb596f2c81e9db', 'request_id': 'req-9870502c-cf5a-4533-9516-b09e14953f85'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:46.982234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7f470b9c694c7fa3eb596f2c81e9db', 'request_id': 'req-9870502c-cf5a-4533-9516-b09e14953f85'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.004500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7f470b9c694c7fa3eb596f2c81e9db', 'request_id': 'req-9870502c-cf5a-4533-9516-b09e14953f85'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.035768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] The token cache is being invalidate because project 3b7f470b9c694c7fa3eb596f2c81e9db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:47.036160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.037545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.040906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 682/1365] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:46 2022] DELETE /identity/v3/projects/3b7f470b9c694c7fa3eb596f2c81e9db => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:47.044917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:47.062549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcff53f-8d93-48ef-aef5-cf637f66860f service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:47.270344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:47.271148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:47.271434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:47.272237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] PATH_INFO: `/v3/users/f945ccb1091c4f3fb53edb2dbc3445c7` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:47.276360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f945ccb1091c4f3fb53edb2dbc3445c7)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:47.278936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:47.399171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f945ccb1091c4f3fb53edb2dbc3445c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.399940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f945ccb1091c4f3fb53edb2dbc3445c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.416589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f945ccb1091c4f3fb53edb2dbc3445c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.418817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f945ccb1091c4f3fb53edb2dbc3445c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.439889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-37de9b69-ea34-4e29-bd0a-d067a0935b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f945ccb1091c4f3fb53edb2dbc3445c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:47.487915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 684/1366] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:47 2022] DELETE /identity/v3/users/f945ccb1091c4f3fb53edb2dbc3445c7 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:48.410052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:48.412376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:48.439737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870502c-cf5a-4533-9516-b09e14953f85 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:48.602366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:48.603043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:48.603457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:48.603607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] PATH_INFO: `/v3/projects/ba9d9648fcc2477bb3eb12b88ccf5c37` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:48.608647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9d9648fcc2477bb3eb12b88ccf5c37)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:48.612757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:48.675205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:48.735945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9d9648fcc2477bb3eb12b88ccf5c37', 'request_id': 'req-88361a87-3ee5-4445-b7dc-54f8ecdea850'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:48.758655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9d9648fcc2477bb3eb12b88ccf5c37', 'request_id': 'req-88361a87-3ee5-4445-b7dc-54f8ecdea850'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:48.767965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9d9648fcc2477bb3eb12b88ccf5c37', 'request_id': 'req-88361a87-3ee5-4445-b7dc-54f8ecdea850'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:48.790354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] The token cache is being invalidate because project ba9d9648fcc2477bb3eb12b88ccf5c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:48.790641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:48.791350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-88361a87-3ee5-4445-b7dc-54f8ecdea850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:48.800568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 683/1367] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:48 2022] DELETE /identity/v3/projects/ba9d9648fcc2477bb3eb12b88ccf5c37 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:48.802652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ca9d5-3bc1-4505-be69-c5efd5d42728 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:48.804114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ca9d5-3bc1-4505-be69-c5efd5d42728 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:48.804570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ca9d5-3bc1-4505-be69-c5efd5d42728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:48.829510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:48.830242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:48.830735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:48.882099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-120ca9d5-3bc1-4505-be69-c5efd5d42728 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:48.917507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-120ca9d5-3bc1-4505-be69-c5efd5d42728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:48.921210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] MFA Rules not processed for user `1c602e0f9e0e4e479a6a3c32febdb14b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:48.922484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:48.928242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:49.111761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 685/1368] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:49.120738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.127897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.129117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.187146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 684/1369] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:49.188682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:49.190871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:49.203529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.205589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:49.215189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 686/1370] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:49.219186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.247954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b3809-70b3-42a5-bc76-8b5fbbd13c74 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.254347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.255484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.256013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.256349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.257265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:49.257721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:49.264215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:49.286559 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 687/1371] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:49.288355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.301806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f94cce-456e-461c-9e9e-67ea4f72d659 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.315795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.316661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.317048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.317414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.318378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:49.321022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:49.381295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 688/1372] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:49 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:49.385484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.431634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0520526-62a0-4e00-9da6-83fd3e967841 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.532083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:49.612889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.613544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.613840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.614111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.650786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8bc20-00c3-4d74-a610-3ecbc0d44ff6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.658166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.658800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.659771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.660063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.661360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:49.664999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:49.718571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 689/1373] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:49 2022] POST /identity/v3/users => generated 288 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:41:49.719761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.765407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1dca-088d-40a7-8f4a-4ac16b83504c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.778596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.780513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.781902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.782422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.785158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:49.787237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:49.828221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 690/1374] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:49.838572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.860382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:49.862884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:49.870915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d6477-9b8b-44c3-a273-18bfb87de97b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.880245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.882102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.884414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.885001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.886421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:49.905520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:49.906713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 685/1375] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:49.937243 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 691/1376] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:49.938900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.953130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ec50-2e8a-422c-8b28-cf87a8c35f86 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:49.968271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b63154-c766-4a62-b396-3b20a6138a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:49.972622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b63154-c766-4a62-b396-3b20a6138a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:49.973044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b63154-c766-4a62-b396-3b20a6138a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:49.973609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b63154-c766-4a62-b396-3b20a6138a4a admin admin] PATH_INFO: `/v3/projects/dc40ce7a6847448790b623a616b996cd/users/0a245d4b24ce47f4a6ba5746b57f87b7/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:49.989668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:49.992175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:50.019063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0a731-92df-4c86-a21f-b86b57028306 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:50.025873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b63154-c766-4a62-b396-3b20a6138a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc40ce7a6847448790b623a616b996cd, user_id=0a245d4b24ce47f4a6ba5746b57f87b7, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.029733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b63154-c766-4a62-b396-3b20a6138a4a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:50.033075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:50.033845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.035761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.036296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] PATH_INFO: `/v3/users/02ec4b80e0d94a47ab53e3052bfe57ab` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.038270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ec4b80e0d94a47ab53e3052bfe57ab)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.042982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:50.079909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 686/1377] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:49 2022] PUT /identity/v3/projects/dc40ce7a6847448790b623a616b996cd/users/0a245d4b24ce47f4a6ba5746b57f87b7/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:50.081194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.081733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.082117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.136061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:50.158227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] MFA Rules not processed for user `0a245d4b24ce47f4a6ba5746b57f87b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:50.162582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:50.166948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ec4b80e0d94a47ab53e3052bfe57ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:50.167971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ec4b80e0d94a47ab53e3052bfe57ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:50.192901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ec4b80e0d94a47ab53e3052bfe57ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:50.193848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ec4b80e0d94a47ab53e3052bfe57ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:50.215446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ed8a5a8f-d246-41b5-a6b1-0cb27520ab61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ec4b80e0d94a47ab53e3052bfe57ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:50.263793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 692/1378] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:49 2022] DELETE /identity/v3/users/02ec4b80e0d94a47ab53e3052bfe57ab => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:50.270226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.270671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.274787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.320015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:50.325359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:50.341280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 687/1379] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:50.346209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:50.351215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:50.461088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:50.479334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 693/1380] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:50.518082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27a204-3b2c-4947-bfb2-a49c03c43365 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:50.530662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:50.531405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.531715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.531984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.532691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.534668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:50.588419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:50.596092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 688/1381] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:50 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:41:50.604806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30877798-bbbe-41e9-8bc5-5124465ea87f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:50.608905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:50.614531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:50.763563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1929ed-e8d3-4825-9a2b-0086c02c18ff admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:50.768459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:50.770754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.772358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:50.772786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.773695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.774627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.775864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.775864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.776084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.780970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:50.797600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.799053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:50.816904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 689/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:41:50.839003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:50.840088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 694/1383] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:50 2022] POST /identity/v3/users => generated 295 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:50.857520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353c593-a370-45d2-beb0-6317a572dda1 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:50.868921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:50.869958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.870458 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.870947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.873313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.875247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:50.894360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 690/1384] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:50.896020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:50.916232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea342fd-4b6a-4669-bd8a-684edee3e27e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:50.927015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:50.928542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:50.929262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:50.929768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] PATH_INFO: `/v3/projects/e9c0ddd655df40ecb1a74118c6aab6cc/users/5f3290a1afd940f1bb13665f2c20a8a3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:50.972477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c0ddd655df40ecb1a74118c6aab6cc, user_id=5f3290a1afd940f1bb13665f2c20a8a3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:50.980597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:51.013448 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 695/1385] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:50 2022] PUT /identity/v3/projects/e9c0ddd655df40ecb1a74118c6aab6cc/users/5f3290a1afd940f1bb13665f2c20a8a3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:41:51.018985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:51.034718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83795a1-e299-4c0d-88b2-09ee9841f047 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:51.148311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856c0cc-5a59-4099-b2e8-e3963547ed88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:51.150173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c0cc-5a59-4099-b2e8-e3963547ed88 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.150494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c0cc-5a59-4099-b2e8-e3963547ed88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.151281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c0cc-5a59-4099-b2e8-e3963547ed88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:51.151945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856c0cc-5a59-4099-b2e8-e3963547ed88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:51.154143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856c0cc-5a59-4099-b2e8-e3963547ed88 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:51.172280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 691/1386] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:41:51.172802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:51.189111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a29f-3760-413d-b18f-a8f31ea8a804 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:51.204387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:51.205455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.205832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.206455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] PATH_INFO: `/v3/projects/e9c0ddd655df40ecb1a74118c6aab6cc/users/5f3290a1afd940f1bb13665f2c20a8a3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:51.216658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c0ddd655df40ecb1a74118c6aab6cc, user_id=5f3290a1afd940f1bb13665f2c20a8a3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:51.227533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:51.244105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:51.262169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 696/1387] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:51 2022] PUT /identity/v3/projects/e9c0ddd655df40ecb1a74118c6aab6cc/users/5f3290a1afd940f1bb13665f2c20a8a3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:51.264722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.265263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.265794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:51.309262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:51.311351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:51.369672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] MFA Rules not processed for user `5f3290a1afd940f1bb13665f2c20a8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:51.371239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:51.487717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:51.504641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93933c-4f9e-4c4b-84b9-eb9dee2f0fea admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:51.524539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:51.525206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.525485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.525758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] PATH_INFO: `/v3/projects/f80702350d5e4a26babbea591f9ae8e2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:51.531765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80702350d5e4a26babbea591f9ae8e2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:51.536551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:51.556326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 692/1388] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:51.570824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:51.573153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:51.681670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80702350d5e4a26babbea591f9ae8e2', 'request_id': 'req-82ecd087-6597-491b-8e23-6f15ef180c7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:51.699870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80702350d5e4a26babbea591f9ae8e2', 'request_id': 'req-82ecd087-6597-491b-8e23-6f15ef180c7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:51.719819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80702350d5e4a26babbea591f9ae8e2', 'request_id': 'req-82ecd087-6597-491b-8e23-6f15ef180c7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:51.745437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] The token cache is being invalidate because project f80702350d5e4a26babbea591f9ae8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:51.745437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:51.746007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-82ecd087-6597-491b-8e23-6f15ef180c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:51.748193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1b5da-4369-47ae-8982-084e2dd03063 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:51.754499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 697/1389] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:51 2022] DELETE /identity/v3/projects/f80702350d5e4a26babbea591f9ae8e2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:51.755758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd7f7-56ff-4a06-9746-138bcbf8445b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.755919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd7f7-56ff-4a06-9746-138bcbf8445b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.756142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd7f7-56ff-4a06-9746-138bcbf8445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:51.813420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a65bd7f7-56ff-4a06-9746-138bcbf8445b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:51.813420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a65bd7f7-56ff-4a06-9746-138bcbf8445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:51.937224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:51.937876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.938156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.938431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:51.944440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:51.989062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 698/1390] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:51.998140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:51.998412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:51.998627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.055798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] MFA Rules not processed for user `f95c1cd09e1042abbb5725f241ce147d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:52.056960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:52.124822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:52.201759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:52.203237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e32bf6-535e-4c42-a838-231caa0a0271 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:52.236789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 693/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:52.238217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:52.238503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:52.238774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.304929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:52.306302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:52.309994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 699/1392] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:52.327306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:52.328905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:52.352887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 694/1393] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:52.354868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:52.375722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39187616-f416-4bf6-8f94-34f82fdeb72e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:52.395242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:52.395897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:52.396175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:52.396445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.397101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:52.397398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:52.410427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:52.450350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 695/1394] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:52.454163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:52.495466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39709c-b2b3-408b-82d4-a88135122911 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:52.504622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:52.505360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:52.505975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:52.506097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.506836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:52.509249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:52.588768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f6c8f-1ee7-42a4-ae1d-b4b776f46ecb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:52.607982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 696/1395] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:52 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:52.611625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:52.612886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:52.613623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:52.613898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:52.614167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.627099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:52.634519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:52.643594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:52.666417 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 700/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:41:52.792797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d1009-7b33-4a33-b44d-bfa25e510ec7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:52.806372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:52.807365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:52.808202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:52.810328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.811284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:52.819485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:52.885194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 697/1397] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:52 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:52.888193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:52.918718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ccc6-d946-43c2-98dd-cc0d0ec0021e service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:52.957976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:52.959015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:52.959561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:52.960110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:52.962154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:52.969533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:52.991250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 701/1398] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:52.994971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:53.008212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54188de2-177c-4ce8-a790-de3cebf80270 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:53.029647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad531a-afc4-4d44-8a86-ee5267d05ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:53.030847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad531a-afc4-4d44-8a86-ee5267d05ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.031312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad531a-afc4-4d44-8a86-ee5267d05ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.031694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad531a-afc4-4d44-8a86-ee5267d05ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.032454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad531a-afc4-4d44-8a86-ee5267d05ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:53.036485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad531a-afc4-4d44-8a86-ee5267d05ee4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:53.069014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 698/1399] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:41:53.073589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:53.099803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cf90e-a2c6-491e-be0f-6a355b00134e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:53.119665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:53.119665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.119665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.119665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] PATH_INFO: `/v3/projects/ad85348d701e432ca0f8777fcc134bae/users/aa7f407234d64999afafe79145d9e36c/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.171244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad85348d701e432ca0f8777fcc134bae, user_id=aa7f407234d64999afafe79145d9e36c, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:53.174155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:53.234358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 702/1400] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:53 2022] PUT /identity/v3/projects/ad85348d701e432ca0f8777fcc134bae/users/aa7f407234d64999afafe79145d9e36c/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:53.246170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3a49-04e9-4381-9dea-b411845f6a97 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.246871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3a49-04e9-4381-9dea-b411845f6a97 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.247191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3a49-04e9-4381-9dea-b411845f6a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.340533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-29ef3a49-04e9-4381-9dea-b411845f6a97 None None] MFA Rules not processed for user `aa7f407234d64999afafe79145d9e36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:53.341725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-29ef3a49-04e9-4381-9dea-b411845f6a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:53.474139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 699/1401] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:53.483803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:53.487052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:53.506315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.507455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.507455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.574151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] MFA Rules not processed for user `2e63c3f83f3d41018e6330f26ed7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:53.576881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:53.719805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626c193-73d5-4610-8a50-f1c87299a10d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:53.733846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:53.735120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.735473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.736771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.747471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 700/1402] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:53.750983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:53.753932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:53.756133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:53.765226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:53.779408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:53.790903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3284e1-6908-488a-a7e6-3679ad44de3a None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:53.796562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 703/1403] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:53.798358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:53.806059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:53.811890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:53.812528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.812810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.813138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.833721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:53.844539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:53.877263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 701/1404] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:41:53.960549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca864b59-7566-42e1-bb59-c44c8c22fce5 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:53.968326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:53.969511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:53.970036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:53.970503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] PATH_INFO: `/v3/projects/cb58f4783188412787989f13a9929332` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:53.972759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb58f4783188412787989f13a9929332)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:53.998463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:54.077033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb58f4783188412787989f13a9929332', 'request_id': 'req-e4d4ec71-6c92-4441-b12d-98f149d6f77d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.092848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb58f4783188412787989f13a9929332', 'request_id': 'req-e4d4ec71-6c92-4441-b12d-98f149d6f77d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.106767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb58f4783188412787989f13a9929332', 'request_id': 'req-e4d4ec71-6c92-4441-b12d-98f149d6f77d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.136636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] The token cache is being invalidate because project cb58f4783188412787989f13a9929332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:54.137138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.139374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.144144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 704/1405] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:53 2022] DELETE /identity/v3/projects/cb58f4783188412787989f13a9929332 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:54.149346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:54.179427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81458ab-1009-4d70-876f-fd6e721d3f55 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:54.186003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:54.188699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:54.293884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:54.294829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:54.295586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:54.295586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] PATH_INFO: `/v3/users/5c1008b183b3451ea9cea7ce6700849e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:54.300242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1008b183b3451ea9cea7ce6700849e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:54.305437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:54.313558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4ec71-6c92-4441-b12d-98f149d6f77d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:54.319917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:54.322663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:54.323187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:54.323687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] PATH_INFO: `/v3/users/aa7f407234d64999afafe79145d9e36c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:54.325386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7f407234d64999afafe79145d9e36c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:54.327966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:54.378952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:54.428326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1008b183b3451ea9cea7ce6700849e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.431860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1008b183b3451ea9cea7ce6700849e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.432226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7f407234d64999afafe79145d9e36c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.433069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7f407234d64999afafe79145d9e36c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.433181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1008b183b3451ea9cea7ce6700849e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.442439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7f407234d64999afafe79145d9e36c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.444681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7f407234d64999afafe79145d9e36c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.451226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1008b183b3451ea9cea7ce6700849e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.456722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2889de48-1d79-45e8-accf-2de961df4be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7f407234d64999afafe79145d9e36c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.474636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1008b183b3451ea9cea7ce6700849e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:54.505439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 705/1406] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:54 2022] DELETE /identity/v3/users/aa7f407234d64999afafe79145d9e36c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:54.511698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:54.511984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:54.512250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:54.518603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 702/1407] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:41:54 2022] DELETE /identity/v3/users/5c1008b183b3451ea9cea7ce6700849e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:54.590072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:54.593490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:54.734051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 706/1408] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:54.922182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:54.924989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:55.029156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ac8a-e1f6-483b-936f-02f0aa5348de admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:55.036870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:55.037820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:55.038208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:55.038583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:55.051487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:55.058378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:55.071871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 703/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:55.885493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:55.887149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:55.911491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd618f-626a-4a22-8767-8a34595605f7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:55.927560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:55.931232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:55.931688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:55.932069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] PATH_INFO: `/v3/projects/ad85348d701e432ca0f8777fcc134bae` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:55.932182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:55.933617 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:55.934173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad85348d701e432ca0f8777fcc134bae)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:55.937874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:55.957902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9a8fd-f005-409f-9c66-815fb6366dc5 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:55.974650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:55.975307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:55.975614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:55.975883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] PATH_INFO: `/v3/projects/e2581ff2ce4143998d30b233bff330df` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:55.984520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2581ff2ce4143998d30b233bff330df)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:55.986703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:56.013258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:56.061091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad85348d701e432ca0f8777fcc134bae', 'request_id': 'req-a55a96ba-4735-4850-bab5-e4ed95d94d58'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.090490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad85348d701e432ca0f8777fcc134bae', 'request_id': 'req-a55a96ba-4735-4850-bab5-e4ed95d94d58'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.111029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad85348d701e432ca0f8777fcc134bae', 'request_id': 'req-a55a96ba-4735-4850-bab5-e4ed95d94d58'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.157104 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2581ff2ce4143998d30b233bff330df', 'request_id': 'req-dbd3e17e-fc33-4806-8e75-066290a0d116'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.170768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] The token cache is being invalidate because project ad85348d701e432ca0f8777fcc134bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:56.171960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.174847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-a55a96ba-4735-4850-bab5-e4ed95d94d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.181211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2581ff2ce4143998d30b233bff330df', 'request_id': 'req-dbd3e17e-fc33-4806-8e75-066290a0d116'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.183850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 707/1410] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:55 2022] DELETE /identity/v3/projects/ad85348d701e432ca0f8777fcc134bae => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:56.187291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ab74f-0425-423c-8d42-436f0444693d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.188954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ab74f-0425-423c-8d42-436f0444693d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.189882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ab74f-0425-423c-8d42-436f0444693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.193153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2581ff2ce4143998d30b233bff330df', 'request_id': 'req-dbd3e17e-fc33-4806-8e75-066290a0d116'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.218985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] The token cache is being invalidate because project e2581ff2ce4143998d30b233bff330df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:41:56.219616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.222713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dbd3e17e-fc33-4806-8e75-066290a0d116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:41:56.225360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 704/1411] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:41:55 2022] DELETE /identity/v3/projects/e2581ff2ce4143998d30b233bff330df => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:56.226820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d48af9-4392-4425-813c-7562c39efa4a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.228221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d48af9-4392-4425-813c-7562c39efa4a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.230822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d48af9-4392-4425-813c-7562c39efa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.269299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c86ab74f-0425-423c-8d42-436f0444693d None None] MFA Rules not processed for user `3d57230baeb24e8a8a511a0c6417077e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:56.273348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c86ab74f-0425-423c-8d42-436f0444693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.282640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-53d48af9-4392-4425-813c-7562c39efa4a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:56.285142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-53d48af9-4392-4425-813c-7562c39efa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.309603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:56.379926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 708/1412] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:56.392773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba791eb-303b-41c4-a0c3-64d21a042261 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.393199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba791eb-303b-41c4-a0c3-64d21a042261 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.393618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba791eb-303b-41c4-a0c3-64d21a042261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.438992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4ba791eb-303b-41c4-a0c3-64d21a042261 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:56.439997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 705/1413] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:56.440783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4ba791eb-303b-41c4-a0c3-64d21a042261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.443985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adb9bf-b111-4c8e-abcc-5f6031a49d79 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.444385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adb9bf-b111-4c8e-abcc-5f6031a49d79 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.444761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27adb9bf-b111-4c8e-abcc-5f6031a49d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.451267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 709/1414] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:56.453957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.454389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.454866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.481199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-27adb9bf-b111-4c8e-abcc-5f6031a49d79 None None] MFA Rules not processed for user `3d57230baeb24e8a8a511a0c6417077e`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:56.482805 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-27adb9bf-b111-4c8e-abcc-5f6031a49d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.484991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:56.488867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.497347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 706/1415] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:56.499648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.500668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.500668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.507269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 710/1416] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:56.509123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.510769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.575991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:56.582844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:56.607385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 707/1417] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:56.608853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.618749 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc505-d0b2-41b4-a1ec-5fc9f53a6a6d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:56.625510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:56.626921 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.627340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.627753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.635574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f86609-6da9-4c1e-8289-66fc2bd1dbd3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:56.646765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:56.651793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:56.658169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:56.659692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.660141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.660550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.661360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:56.661807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:56.668553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:56.674653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 711/1418] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:56.676863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.694331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 708/1419] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:56.697713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.699877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f49ac-97b0-4be4-bcf2-dcc6ee37444c service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:56.711485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:56.721636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.722063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.722347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.723035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:41:56.723344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:56.740442 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94d83b-7fc3-41bf-b2d0-044e5b1ee434 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:56.742782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:56.758044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:56.758797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.759579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.759579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.760384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:56.764903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:56.776027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 712/1420] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:41:56.787114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.812959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66755c-cb8d-4e33-9ca7-fec27eb7f365 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:56.827168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:56.827828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:56.828105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:56.828381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:56.829017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:56.830945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:56.856071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 709/1421] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:56.857382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.948418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 713/1422] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:56.951214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:56.983213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:57.025674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58ed1e-8bcd-487f-859d-c3e39c49bbb4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.153948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:57.161368 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ee12f-e741-4832-ac3c-5c6bab101643 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.165597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.166556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.166833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.167099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.167844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.173762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.177029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.179880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.180223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.180557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.181277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.183328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.225190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 710/1423] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/users => generated 287 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:57.229795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.237408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 714/1424] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:56 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:41:57.242004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.251591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801345ed-974f-4c9c-9e0d-17443c1ebd68 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.263674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e18f8-617c-4bf6-a323-d65da48469f8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.268245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.268877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.269186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.269489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.270086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.272228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.291897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.295795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.296851 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.297317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.299509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.301977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.304291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 711/1425] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:41:57.308391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.343542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 715/1426] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:41:57.345528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fb2a-deab-4026-9c0d-fbf8e54552d6 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.347818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.361091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.361642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.361921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.362194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.362843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.364456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1242-46a8-4bf2-8385-995afd0a5199 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.364577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.373664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.374638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.375108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.384454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.385391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.387427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.395165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 712/1427] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:57.399883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.418682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24a68-af7f-4ccc-a3e1-96674fde16ba admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.425704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 716/1428] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:57.426655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.432390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b83b2c-5ddf-4c01-ad6f-4ad47e845106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.433468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83b2c-5ddf-4c01-ad6f-4ad47e845106 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.433930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83b2c-5ddf-4c01-ad6f-4ad47e845106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.434489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83b2c-5ddf-4c01-ad6f-4ad47e845106 admin admin] PATH_INFO: `/v3/projects/2791cf70e78a453888438f37b9564896/users/0cd91022635647bbbb3324818a5e04d4/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.469300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9993a1-a26e-4dc2-ae77-77a3f01fcf9b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.472204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b83b2c-5ddf-4c01-ad6f-4ad47e845106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2791cf70e78a453888438f37b9564896, user_id=0cd91022635647bbbb3324818a5e04d4, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.475740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b83b2c-5ddf-4c01-ad6f-4ad47e845106 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.487975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce397643-09c9-45ca-ab6c-f1d5294b7708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.488640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce397643-09c9-45ca-ab6c-f1d5294b7708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.488917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce397643-09c9-45ca-ab6c-f1d5294b7708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.489185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce397643-09c9-45ca-ab6c-f1d5294b7708 admin admin] PATH_INFO: `/v3/projects/5fcd567ba5834cf5ab17aae87b1c2686/users/a88210e2bbaa412dabe42a8f25b7a4bd/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.509859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 713/1429] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:57 2022] PUT /identity/v3/projects/2791cf70e78a453888438f37b9564896/users/0cd91022635647bbbb3324818a5e04d4/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:41:57.521410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06455a00-2640-455b-85e8-fe5e000e8415 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.521700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06455a00-2640-455b-85e8-fe5e000e8415 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.522074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06455a00-2640-455b-85e8-fe5e000e8415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.537159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce397643-09c9-45ca-ab6c-f1d5294b7708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcd567ba5834cf5ab17aae87b1c2686, user_id=a88210e2bbaa412dabe42a8f25b7a4bd, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.540748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce397643-09c9-45ca-ab6c-f1d5294b7708 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.566439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 717/1430] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:57 2022] PUT /identity/v3/projects/5fcd567ba5834cf5ab17aae87b1c2686/users/a88210e2bbaa412dabe42a8f25b7a4bd/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:57.584274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3540ba3-23af-4436-9d80-79a68232f00a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.586679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3540ba3-23af-4436-9d80-79a68232f00a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.588382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3540ba3-23af-4436-9d80-79a68232f00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.589569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-06455a00-2640-455b-85e8-fe5e000e8415 None None] MFA Rules not processed for user `0cd91022635647bbbb3324818a5e04d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:57.590962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-06455a00-2640-455b-85e8-fe5e000e8415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:57.672245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a3540ba3-23af-4436-9d80-79a68232f00a None None] MFA Rules not processed for user `a88210e2bbaa412dabe42a8f25b7a4bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:57.676109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a3540ba3-23af-4436-9d80-79a68232f00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:57.696438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 714/1431] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:57.698262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1dc13-888f-4e20-9817-5382e696fea3 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.698896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1dc13-888f-4e20-9817-5382e696fea3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.699536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1dc13-888f-4e20-9817-5382e696fea3 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.702103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 715/1432] 10.222.0.39 () {54 vars in 912 bytes} [Tue Oct 11 06:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:41:57.703081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1dc13-888f-4e20-9817-5382e696fea3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:57.766396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:57.835120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 718/1433] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:57.839669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.839822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.840010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.860766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1dc13-888f-4e20-9817-5382e696fea3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:57.876995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:57.877945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:57.878804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:57.889904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:57.896299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:57.921177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:57.968130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:57.970450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:58.021875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 716/1434] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:57 2022] POST /identity/v3/projects => generated 371 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:58.023693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.124564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 719/1435] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:41:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:58.126457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.130759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:58.189696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84060d1-864c-47b6-b136-7569401fc2d5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.196061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.197127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.197625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.198075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.199931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.203219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.214747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:58.257558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 717/1436] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:41:58 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:58.259150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.275393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de385a-55d2-4c24-9848-a258682bef40 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.378594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617c3c-67e9-4d70-88f5-af63503d69a9 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.416090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.417513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.419537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.420211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.422735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:58.441634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.442964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.443400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.444907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.446784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.464142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.565275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 718/1437] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:58 2022] POST /identity/v3/users => generated 292 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:58.568782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.572176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:58.596831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.600383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.637306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 720/1438] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:58.640415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.680831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:58.700094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9598-b215-450d-bcd7-e63a973e1168 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.720698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.721742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.723810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.724321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.725194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.734597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.765983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d218-074c-448b-96f8-f203231dad0a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.779342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810429df-3235-4a1a-aedd-bb0359671319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.780376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810429df-3235-4a1a-aedd-bb0359671319 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.780653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810429df-3235-4a1a-aedd-bb0359671319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.780930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810429df-3235-4a1a-aedd-bb0359671319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.784497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 721/1439] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:41:58 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:41:58.790874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.815812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810429df-3235-4a1a-aedd-bb0359671319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.818096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4825117-a842-4c4a-8353-84ad11da3759 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.819939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810429df-3235-4a1a-aedd-bb0359671319 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.836526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.845287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.845900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 719/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:41:58.846309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.847823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.848744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.849189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.849486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.849767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.866407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.895185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 722/1441] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:58.898048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:58.899889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] MFA Rules not processed for user `1c602e0f9e0e4e479a6a3c32febdb14b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:58.901180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:58.925400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f7633-70aa-4b5d-a3dd-a2079f67be3b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:58.940210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:58.941023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:58.941455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:58.942836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:58.944194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:58.947661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:58.986134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 723/1442] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:58.987814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:59.028628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe943dda-36bf-40a6-a2be-5d03f6e24509 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:59.036812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 720/1443] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:59.036812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:59.052446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:59.052773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.053054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.053329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] PATH_INFO: `/v3/projects/be1d333030914033afb2b69d31e6e14b/users/8afde52b50d6482eb37d9a9e4d91d098/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.083737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea07c7-d258-4b9c-9948-96569cdd6821 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:59.100885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1d333030914033afb2b69d31e6e14b, user_id=8afde52b50d6482eb37d9a9e4d91d098, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:59.104145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:59.125087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:59.130326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.131176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.131983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] PATH_INFO: `/v3/projects/ba04331b4e4c4b1aa4a4dd2428901c1e/users/0493f1ab376741728e4afbc9697e64f3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.134675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 724/1444] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:58 2022] PUT /identity/v3/projects/be1d333030914033afb2b69d31e6e14b/users/8afde52b50d6482eb37d9a9e4d91d098/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:59.140762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:59.191062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba04331b4e4c4b1aa4a4dd2428901c1e, user_id=0493f1ab376741728e4afbc9697e64f3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:59.202575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:59.220106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:59.251711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 721/1445] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:59 2022] PUT /identity/v3/projects/ba04331b4e4c4b1aa4a4dd2428901c1e/users/0493f1ab376741728e4afbc9697e64f3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:59.255994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:59.301115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02783886-b396-4c90-a4e7-d58663e5fb33 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:59.472175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47356cd-af48-45c7-b314-a09b222d0f5c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:59.482852 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee5018-cb83-4bd3-9e32-56408d13768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:59.484458 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5018-cb83-4bd3-9e32-56408d13768a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.484752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5018-cb83-4bd3-9e32-56408d13768a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.485028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5018-cb83-4bd3-9e32-56408d13768a admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.485700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee5018-cb83-4bd3-9e32-56408d13768a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:59.487318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee5018-cb83-4bd3-9e32-56408d13768a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:59.513384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:59.514017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.514299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.514634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.515540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:59.520803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 722/1446] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:59.525053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:59.526618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25552bb2-5886-4fa2-90e6-7480a01f31b1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.526806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25552bb2-5886-4fa2-90e6-7480a01f31b1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.527092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25552bb2-5886-4fa2-90e6-7480a01f31b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.553900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 725/1447] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:41:59.555702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:59.560048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-25552bb2-5886-4fa2-90e6-7480a01f31b1 None None] MFA Rules not processed for user `f95c1cd09e1042abbb5725f241ce147d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:59.561273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-25552bb2-5886-4fa2-90e6-7480a01f31b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:59.580728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf783c-be0f-4754-8a5a-5fe3e2109274 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:59.591014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:59.591834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.592185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.592532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] PATH_INFO: `/v3/projects/ba04331b4e4c4b1aa4a4dd2428901c1e/users/0493f1ab376741728e4afbc9697e64f3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.595823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba04331b4e4c4b1aa4a4dd2428901c1e, user_id=0493f1ab376741728e4afbc9697e64f3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:59.598972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:59.638199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 726/1448] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:59 2022] PUT /identity/v3/projects/ba04331b4e4c4b1aa4a4dd2428901c1e/users/0493f1ab376741728e4afbc9697e64f3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:59.639668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:41:59.705099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:41:59.733085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 723/1449] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:59.735016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.735451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.736365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.791571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] MFA Rules not processed for user `0493f1ab376741728e4afbc9697e64f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:59.792957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:59.801662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b74502-7522-4093-b102-f6ea709c1354 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:41:59.814812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39770b5-fe7d-4fa8-a794-a58aa722e836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:41:59.815728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39770b5-fe7d-4fa8-a794-a58aa722e836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.815968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39770b5-fe7d-4fa8-a794-a58aa722e836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.816289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39770b5-fe7d-4fa8-a794-a58aa722e836 admin admin] PATH_INFO: `/v3/projects/be1d333030914033afb2b69d31e6e14b/users/8afde52b50d6482eb37d9a9e4d91d098/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.823137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39770b5-fe7d-4fa8-a794-a58aa722e836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1d333030914033afb2b69d31e6e14b, user_id=8afde52b50d6482eb37d9a9e4d91d098, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:41:59.833052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39770b5-fe7d-4fa8-a794-a58aa722e836 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:41:59.858601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 727/1450] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:41:59 2022] PUT /identity/v3/projects/be1d333030914033afb2b69d31e6e14b/users/8afde52b50d6482eb37d9a9e4d91d098/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:41:59.869312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:41:59.869596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:41:59.869872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:41:59.928991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] MFA Rules not processed for user `8afde52b50d6482eb37d9a9e4d91d098`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:41:59.930301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:41:59.972915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 724/1451] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:41:59.976093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:00.013872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:00.079970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 728/1452] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:41:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:00.094466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:00.096017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:00.115544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650fa45-40cb-4ce1-83ef-959163289f50 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:00.204674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d433a42-20cd-4602-a450-7675359c217a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:00.205535 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d433a42-20cd-4602-a450-7675359c217a service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.205868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d433a42-20cd-4602-a450-7675359c217a service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.206200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d433a42-20cd-4602-a450-7675359c217a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.276156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e52ae8-8b76-41f5-b8a1-3aecc3419c61 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:00.304517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:00.305209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.309253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.309662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.330715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:00.341888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:00.344280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:00.373162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d433a42-20cd-4602-a450-7675359c217a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:00.374938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d433a42-20cd-4602-a450-7675359c217a service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:00.388327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 729/1453] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:00.396765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 725/1454] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:41:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:00.410985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:00.412419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:00.472011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.472294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.472697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.537089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] MFA Rules not processed for user `e97edc31dd834fd8811ad1bb25f1168b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:00.538038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14af8a8-9284-4e96-aadf-6bcb167e3957 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:00.541470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:00.544715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:00.545534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.545965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.546550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.548583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:00.598493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:00.600903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:00.614267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 730/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:00.615220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:00.666950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 726/1456] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:00.668254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:00.687739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad0353-a639-40b1-ae7b-1eb8782a3852 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:00.693118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44459d-98ad-42de-8491-4893ce1674c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:00.694034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44459d-98ad-42de-8491-4893ce1674c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.694853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44459d-98ad-42de-8491-4893ce1674c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.695217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44459d-98ad-42de-8491-4893ce1674c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.701663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1ddd8-cbb9-494c-addc-aab4d325cae8 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:00.709530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:00.710281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.711231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.711649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] PATH_INFO: `/v3/users/a88210e2bbaa412dabe42a8f25b7a4bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.712992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88210e2bbaa412dabe42a8f25b7a4bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:00.716244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:00.737139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:00.797437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44459d-98ad-42de-8491-4893ce1674c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:00.802707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44459d-98ad-42de-8491-4893ce1674c0 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:00.817144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 727/1457] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:00.817963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88210e2bbaa412dabe42a8f25b7a4bd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:00.818798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88210e2bbaa412dabe42a8f25b7a4bd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:00.819481 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1699-0019-4347-bbbf-b10a6e013b17 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.820122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1699-0019-4347-bbbf-b10a6e013b17 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.820793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1699-0019-4347-bbbf-b10a6e013b17 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.823539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 728/1458] 10.222.0.39 () {54 vars in 912 bytes} [Tue Oct 11 06:42:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:42:00.825145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e450-34c3-40cc-b20a-75ca7fd6e78e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.825593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e450-34c3-40cc-b20a-75ca7fd6e78e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.826383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e450-34c3-40cc-b20a-75ca7fd6e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.828128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88210e2bbaa412dabe42a8f25b7a4bd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:00.829065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88210e2bbaa412dabe42a8f25b7a4bd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:00.839968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e64e7092-d0f9-49a4-b1ef-990efdf6e82c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88210e2bbaa412dabe42a8f25b7a4bd'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:00.856949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2188e450-34c3-40cc-b20a-75ca7fd6e78e None None] MFA Rules not processed for user `2e63c3f83f3d41018e6330f26ed7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:00.858163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2188e450-34c3-40cc-b20a-75ca7fd6e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:00.895737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 731/1459] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:00 2022] DELETE /identity/v3/users/a88210e2bbaa412dabe42a8f25b7a4bd => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:00.898745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.899039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.899849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.961232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] MFA Rules not processed for user `e97edc31dd834fd8811ad1bb25f1168b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:00.962478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:00.963249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 729/1460] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:00.969816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fa7d-04b3-452e-99a3-3511373a54fb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:00.970132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fa7d-04b3-452e-99a3-3511373a54fb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:00.970580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fa7d-04b3-452e-99a3-3511373a54fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:00.999875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3836fa7d-04b3-452e-99a3-3511373a54fb None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:01.001712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3836fa7d-04b3-452e-99a3-3511373a54fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:01.016588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:01.141545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 730/1461] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:42:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:01.144420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916d2de-fec9-4d28-97b8-7a4a9487e6fa None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:01.147241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916d2de-fec9-4d28-97b8-7a4a9487e6fa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:01.147803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916d2de-fec9-4d28-97b8-7a4a9487e6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:01.185131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4916d2de-fec9-4d28-97b8-7a4a9487e6fa None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:01.190598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4916d2de-fec9-4d28-97b8-7a4a9487e6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:01.207933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 732/1462] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:01.208728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:01.210053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:01.243194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f8a71-6995-4898-80ac-31fcd20f4507 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:01.248333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:01.248963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:01.249242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:01.249819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:01.259991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:01.261519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:01.273111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 733/1463] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:01.352646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 731/1464] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:01.369868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:01.371638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:01.416846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202e7e0-4045-4656-9d70-04fa205e959e service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:01.429469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:01.430254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:01.430541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:01.430816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:01.464666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:01.471797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:01.502648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 734/1465] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:01.763905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:01.765785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:01.766252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:01.816238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] MFA Rules not processed for user `3d57230baeb24e8a8a511a0c6417077e`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:01.818694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:01.988765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:01.990613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:02.008800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 732/1466] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:02.151953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:02.158673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf847b-0fb4-4f02-b128-10082f2b99fc service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:02.168298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:02.169876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:02.170388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:02.171047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:02.174506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:02.193129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:02.197462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:02.216939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 735/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:02.240395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:02.242731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:02.271808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941847-cd69-493a-add3-7fe1260c30dc None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:02.293326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:02.295598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:02.295914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:02.296295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] PATH_INFO: `/v3/projects/5fcd567ba5834cf5ab17aae87b1c2686` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:02.298737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fcd567ba5834cf5ab17aae87b1c2686)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:02.301886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:02.424523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fcd567ba5834cf5ab17aae87b1c2686', 'request_id': 'req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.463185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fcd567ba5834cf5ab17aae87b1c2686', 'request_id': 'req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.473497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fcd567ba5834cf5ab17aae87b1c2686', 'request_id': 'req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.489051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:02.501970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] The token cache is being invalidate because project 5fcd567ba5834cf5ab17aae87b1c2686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:02.502761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.504319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.513762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 733/1468] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:02 2022] DELETE /identity/v3/projects/5fcd567ba5834cf5ab17aae87b1c2686 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:02.514956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:02.533494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ae2dd-f67d-4a02-993c-413767716ba1 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:02.723797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:02.724493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:02.724493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:02.725658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] PATH_INFO: `/v3/users/0493f1ab376741728e4afbc9697e64f3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:02.729043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0493f1ab376741728e4afbc9697e64f3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:02.731359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:02.839696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0493f1ab376741728e4afbc9697e64f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.841025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0493f1ab376741728e4afbc9697e64f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.859714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0493f1ab376741728e4afbc9697e64f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.861311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0493f1ab376741728e4afbc9697e64f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.888832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f743bdc0-3110-41e0-8852-f16766c692b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0493f1ab376741728e4afbc9697e64f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:02.920386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 736/1469] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:02 2022] DELETE /identity/v3/users/0493f1ab376741728e4afbc9697e64f3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:03.769374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:03.770893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:03.811977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb9252-1a6c-4cc2-b4ee-ce884dbe2dd4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:04.085800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:04.086530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:04.088208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:04.088491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] PATH_INFO: `/v3/projects/ba04331b4e4c4b1aa4a4dd2428901c1e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:04.095037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba04331b4e4c4b1aa4a4dd2428901c1e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:04.104485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:04.260982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba04331b4e4c4b1aa4a4dd2428901c1e', 'request_id': 'req-4d28af99-6f82-4302-914b-a1ebdd2ee698'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:04.286993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba04331b4e4c4b1aa4a4dd2428901c1e', 'request_id': 'req-4d28af99-6f82-4302-914b-a1ebdd2ee698'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:04.323607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba04331b4e4c4b1aa4a4dd2428901c1e', 'request_id': 'req-4d28af99-6f82-4302-914b-a1ebdd2ee698'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:04.364418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] The token cache is being invalidate because project ba04331b4e4c4b1aa4a4dd2428901c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:04.364542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:04.366128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4d28af99-6f82-4302-914b-a1ebdd2ee698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:04.383792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 734/1470] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:03 2022] DELETE /identity/v3/projects/ba04331b4e4c4b1aa4a4dd2428901c1e => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:04.387791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:04.388301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:04.388376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:04.478710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:04.485842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:04.802321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 737/1471] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:04.811238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:04.812271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:04.812767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:04.882712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:04.885274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:04.908412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 735/1472] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:04.912143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:04.951021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbb61-7988-412e-9d6e-c44a388105a3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:04.965869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:04.966704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:04.967689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:04.967979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:04.969728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:04.970038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:04.979015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:05.020393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:05.042785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 738/1473] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:05.046478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:05.065120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03618e12-b057-4ca9-9729-87ecb5a234a6 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:05.080556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:05.081751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.082300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.082719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.083595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:05.086456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:05.151525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 736/1474] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:05 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:05.155947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:05.175900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa377e4-6fd2-441e-b05d-8434ee31f0e6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:05.282196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:05.282196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.282196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.282196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.282642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:05.285287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:05.317182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 739/1475] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:05 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:05.321005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:05.333222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea51d1-8205-4aa0-a0aa-06aac7257b6d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:05.348303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:05.350114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.352117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.352669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.356052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:05.360928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:05.391965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 737/1476] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:05.395087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:05.427485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2e6e8-4178-42c8-8e19-0dd9532e25a3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:05.439089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b77098-b5fe-4df7-a564-db1309fd8889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:05.440596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b77098-b5fe-4df7-a564-db1309fd8889 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.441051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b77098-b5fe-4df7-a564-db1309fd8889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.441594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b77098-b5fe-4df7-a564-db1309fd8889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.442401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b77098-b5fe-4df7-a564-db1309fd8889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:05.444252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b77098-b5fe-4df7-a564-db1309fd8889 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:05.462511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 740/1477] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:05.466602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:05.480282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4781af-c284-4b2f-859a-6f1aad0b968c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:05.493196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91deb10a-bf17-4de7-8910-51e8f447a3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:05.493835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deb10a-bf17-4de7-8910-51e8f447a3cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.494119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deb10a-bf17-4de7-8910-51e8f447a3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.494399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deb10a-bf17-4de7-8910-51e8f447a3cc admin admin] PATH_INFO: `/v3/projects/55605685384742a69018071bf59cefef/users/feeadf34e2824e0aa97da7a8dd9631fe/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.555741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deb10a-bf17-4de7-8910-51e8f447a3cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=55605685384742a69018071bf59cefef, user_id=feeadf34e2824e0aa97da7a8dd9631fe, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:05.558723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deb10a-bf17-4de7-8910-51e8f447a3cc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:05.604221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 738/1478] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:05 2022] PUT /identity/v3/projects/55605685384742a69018071bf59cefef/users/feeadf34e2824e0aa97da7a8dd9631fe/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:05.606174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.606274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.606536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.797859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] MFA Rules not processed for user `feeadf34e2824e0aa97da7a8dd9631fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:05.799107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:05.841127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:05.841660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:05.842131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:05.985679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:06.053309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:06.055332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:06.086923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 741/1479] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:06.088191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:06.319917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 739/1480] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:42:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:06.390066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4eff0-dc60-4969-b1e7-87143bcdda86 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:06.402328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:06.403847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:06.404270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:06.404791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:06.405835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:06.411560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:06.457474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:06.526700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 742/1481] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:06 2022] POST /identity/v3/projects => generated 381 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:06.536907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:06.560474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f14530-c893-4c0c-bc4d-ec08b5ab1758 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:06.708624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:06.711130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:06.727045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:06.727949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:06.728138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:06.728411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:06.729063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:06.738272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:06.831701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 740/1482] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:06 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:06.832413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:06.870436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a39819-4d69-4daf-9b7f-010eefdf3374 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:06.905982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:06.910661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:06.912444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:06.912903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:06.918552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:06.926713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:06.959138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 741/1483] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:06.970959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:06.997749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9cf3e-afbc-421d-9997-b3ff2ac416d4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:07.008566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:07.009733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:07.010152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:07.011119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] PATH_INFO: `/v3/projects/0f3f9536a7a14e2697c6c9f471da7b11/users/4e3e99c9ba7d42b5a74e3c2e8428c535/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:07.045159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75396b3e-7b55-40f9-8f37-69eea6ec30a4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:07.052141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:07.052974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:07.053526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:07.056510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3f9536a7a14e2697c6c9f471da7b11, user_id=4e3e99c9ba7d42b5a74e3c2e8428c535, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:07.059911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:07.062336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:07.095524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 742/1484] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:06 2022] PUT /identity/v3/projects/0f3f9536a7a14e2697c6c9f471da7b11/users/4e3e99c9ba7d42b5a74e3c2e8428c535/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:07.096636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:07.219602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:07.226662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:07.238780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:07.254257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 743/1485] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:42:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:42:07.256225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5fd6f-92d9-423e-8aaa-abce78cbc705 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:07.261671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1685-858d-422f-a763-d24e7a6991aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:07.262439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1685-858d-422f-a763-d24e7a6991aa admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:07.263245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1685-858d-422f-a763-d24e7a6991aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:07.263448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1685-858d-422f-a763-d24e7a6991aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:07.264329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d1685-858d-422f-a763-d24e7a6991aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:07.266789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d1685-858d-422f-a763-d24e7a6991aa admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:07.286195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 743/1486] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:07.290943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:07.323571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314e031-57b8-42db-82eb-5f34ca29b5ad service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:07.344454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:07.349781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:07.352400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:07.354163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] PATH_INFO: `/v3/projects/0f3f9536a7a14e2697c6c9f471da7b11/users/4e3e99c9ba7d42b5a74e3c2e8428c535/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:07.361942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3f9536a7a14e2697c6c9f471da7b11, user_id=4e3e99c9ba7d42b5a74e3c2e8428c535, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:07.369198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:07.407169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 744/1487] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:07 2022] PUT /identity/v3/projects/0f3f9536a7a14e2697c6c9f471da7b11/users/4e3e99c9ba7d42b5a74e3c2e8428c535/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:07.418417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:07.419089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:07.419694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:07.680266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] MFA Rules not processed for user `4e3e99c9ba7d42b5a74e3c2e8428c535`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:07.683728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:07.861140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 744/1488] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:07.875706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:07.877479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:08.004505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2628f-5298-4d25-b28b-d1fe0533b282 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:08.009561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:08.010268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:08.010616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:08.010950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:08.021578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:08.023980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:08.035816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 745/1489] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:08.509529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:08.518949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292eccc-4d74-448c-9590-fc9cfb45d8f2 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:08.523651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3e99c9ba7d42b5a74e3c2e8428c535', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f3f9536a7a14e2697c6c9f471da7b11', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:08.524272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:08.524551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:08.524824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:08.525491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:08.525795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:08.527397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:08.543537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 745/1490] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:08.544695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:08.549051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:08.555134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573fa07-59a5-4c16-ad1a-2ec43de27371 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:08.561358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3e99c9ba7d42b5a74e3c2e8428c535', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f3f9536a7a14e2697c6c9f471da7b11', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:08.562438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:08.562723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:08.562998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:08.563700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:08.565768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:08.595486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 746/1491] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:08 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:08.599861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:08.610465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7426f-ca8a-40e1-95b6-cf06c9bbbb52 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:08.671466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3e99c9ba7d42b5a74e3c2e8428c535', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f3f9536a7a14e2697c6c9f471da7b11', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:08.672090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:08.672376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:08.672652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:08.673305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:08.673613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:08.675197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:08.696325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 746/1492] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:42:08.749650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:08.775057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22385d0d-86ae-49da-a0dc-df52dd4a1bc3 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:08.780612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e3e99c9ba7d42b5a74e3c2e8428c535', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0f3f9536a7a14e2697c6c9f471da7b11', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:08.781620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:08.782089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:08.782510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] PATH_INFO: `/v3/projects/d59b341c3f36425f8bfacb0133959297` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:08.784035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d59b341c3f36425f8bfacb0133959297)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:08.786113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:08.869343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59b341c3f36425f8bfacb0133959297', 'request_id': 'req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:08.877812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59b341c3f36425f8bfacb0133959297', 'request_id': 'req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:08.888755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59b341c3f36425f8bfacb0133959297', 'request_id': 'req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:08.914452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] The token cache is being invalidate because project d59b341c3f36425f8bfacb0133959297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:08.914941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:08.916139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-80d4a17f-12e2-40dd-bf76-750cc6c4b0d5 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:08.918406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 747/1493] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:08 2022] DELETE /identity/v3/projects/d59b341c3f36425f8bfacb0133959297 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:09.127632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:09.129546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:09.219448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:09.239017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4323c-4ad4-4005-a6e6-2f91afe91954 tempest-VolumeQuotaClassesTest-1890623074 tempest-VolumeQuotaClassesTest-1890623074-project-admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:09.385578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:09.386318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:09.386502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:09.386778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] PATH_INFO: `/v3/users/feeadf34e2824e0aa97da7a8dd9631fe` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:09.388039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feeadf34e2824e0aa97da7a8dd9631fe)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:09.390752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:09.507840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feeadf34e2824e0aa97da7a8dd9631fe'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:09.508557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feeadf34e2824e0aa97da7a8dd9631fe'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:09.509182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feeadf34e2824e0aa97da7a8dd9631fe'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:09.527559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feeadf34e2824e0aa97da7a8dd9631fe'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:09.546155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feeadf34e2824e0aa97da7a8dd9631fe'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:09.595274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 747/1494] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:09 2022] DELETE /identity/v3/users/feeadf34e2824e0aa97da7a8dd9631fe => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:09.599947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:09.601702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:09.602277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:09.672626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:09.678263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:09.815115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 748/1495] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:09.825930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:09.827386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:09.938850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab5e91-10aa-45bc-b155-20927d417b96 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:09.950505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:09.951142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:09.953140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:09.953439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:09.977111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:09.981396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:10.017376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 748/1496] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:10.509096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:10.510681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:10.529747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fbf03-283e-4228-a41e-8a214d6c37fc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:10.543057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:10.543057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:10.543057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:10.543702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] PATH_INFO: `/v3/projects/55605685384742a69018071bf59cefef` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:10.545076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55605685384742a69018071bf59cefef)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:10.547116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:10.633354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55605685384742a69018071bf59cefef', 'request_id': 'req-f3315a1e-36b1-489f-a32a-0aafe3ec8402'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:10.647024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55605685384742a69018071bf59cefef', 'request_id': 'req-f3315a1e-36b1-489f-a32a-0aafe3ec8402'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:10.652616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:10.660205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55605685384742a69018071bf59cefef', 'request_id': 'req-f3315a1e-36b1-489f-a32a-0aafe3ec8402'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:10.691894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] The token cache is being invalidate because project 55605685384742a69018071bf59cefef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:10.695832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:10.697647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:10.701340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 749/1497] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:10 2022] DELETE /identity/v3/projects/55605685384742a69018071bf59cefef => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:10.705569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:10.730445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034d0cb-3b47-4da5-b25e-84f8305dca82 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:10.790666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:10.891953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:10.892746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:10.893231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:10.893658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] PATH_INFO: `/v3/users/4e3e99c9ba7d42b5a74e3c2e8428c535` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:10.896670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3e99c9ba7d42b5a74e3c2e8428c535)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:10.905520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:11.009054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3e99c9ba7d42b5a74e3c2e8428c535'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:11.010490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3e99c9ba7d42b5a74e3c2e8428c535'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:11.018433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3e99c9ba7d42b5a74e3c2e8428c535'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:11.035809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3e99c9ba7d42b5a74e3c2e8428c535'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:11.048641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f1ff1ff2-56fc-4204-913f-9977b5142bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3e99c9ba7d42b5a74e3c2e8428c535'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:11.072530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 749/1498] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:10 2022] DELETE /identity/v3/users/4e3e99c9ba7d42b5a74e3c2e8428c535 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:11.736264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:11.738627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:11.761260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3315a1e-36b1-489f-a32a-0aafe3ec8402 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:11.930625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:11.931283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:11.931604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:11.931867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] PATH_INFO: `/v3/projects/0f3f9536a7a14e2697c6c9f471da7b11` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:11.933254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3f9536a7a14e2697c6c9f471da7b11)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:11.935155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.056610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3f9536a7a14e2697c6c9f471da7b11', 'request_id': 'req-f6d00281-8ea8-46df-a844-bc160564cac4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:12.064912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3f9536a7a14e2697c6c9f471da7b11', 'request_id': 'req-f6d00281-8ea8-46df-a844-bc160564cac4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:12.076152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3f9536a7a14e2697c6c9f471da7b11', 'request_id': 'req-f6d00281-8ea8-46df-a844-bc160564cac4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:12.090471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] The token cache is being invalidate because project 0f3f9536a7a14e2697c6c9f471da7b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:12.090772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:12.091528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f6d00281-8ea8-46df-a844-bc160564cac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:12.095453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 750/1499] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:11 2022] DELETE /identity/v3/projects/0f3f9536a7a14e2697c6c9f471da7b11 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:12.098478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.098997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.099092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.130403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:12.131949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:12.222291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 750/1500] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:12.230237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.230521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.230803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.264685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:12.266129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:12.274735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 751/1501] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:12.279012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.290686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473576bb-10a7-4d20-b6b2-910720041667 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.296214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.297022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.297472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.297896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.298857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:12.299313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.301483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.321767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 751/1502] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:12.323408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.336320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c692a-bddf-481f-8deb-e75a7c91f4d0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.341936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.342545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.342826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.343101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.343785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.345780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.382252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 752/1503] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:12.386521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.398550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51935e91-ac49-4468-814f-c93ddfc6615d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.453289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:12.501191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.501803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.502083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.502352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.502998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.504994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.531169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 752/1504] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:12.535263 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.546037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834badbf-d62f-440c-aa7f-f1e7817aa48e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.550610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.551221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.551543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.551819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.552463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.554680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.565217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 753/1505] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:12.569443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.580931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4cd4-fe98-49d8-ad27-44ac6d85e9a0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.587714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5644-43d4-4f88-8ab7-5f25fd926b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.588490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5644-43d4-4f88-8ab7-5f25fd926b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.588777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5644-43d4-4f88-8ab7-5f25fd926b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.589117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5644-43d4-4f88-8ab7-5f25fd926b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.589921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d5644-43d4-4f88-8ab7-5f25fd926b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.591602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d5644-43d4-4f88-8ab7-5f25fd926b07 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.603615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 753/1506] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:12.608092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.620640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874996-e908-47be-be07-fb483cda9480 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.626682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.627449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.627727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.628003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] PATH_INFO: `/v3/projects/fe76e86641ab4a6d95643c9a7b83debe/users/480d50c41505423eaaf9c9653d1367e1/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.647284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe76e86641ab4a6d95643c9a7b83debe, user_id=480d50c41505423eaaf9c9653d1367e1, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.650774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.666436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 754/1507] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:12 2022] PUT /identity/v3/projects/fe76e86641ab4a6d95643c9a7b83debe/users/480d50c41505423eaaf9c9653d1367e1/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:12.678323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.678602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.678894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.707087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] MFA Rules not processed for user `480d50c41505423eaaf9c9653d1367e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:12.708379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:12.786531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 754/1508] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:12.791755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.807452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:12.871135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3167f-95bc-4d5c-b9ea-1254a727b155 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:12.876340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:12.876958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:12.877267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:12.877538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:12.878211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:12.880259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:12.920939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 755/1509] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:12.925592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:12.941843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002bc86-d2de-4354-8f7d-e48722fc5f70 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:13.064935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:13.065572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.065854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.066160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.066803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:13.069254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:13.102021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 755/1510] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:12 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:13.106258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:13.121603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e9531-0088-4653-ab62-378d78baff05 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:13.141881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:13.142522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.142896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.143174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.143865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:13.146171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:13.199992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 756/1511] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:13.202492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:13.214643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3349d9-8225-4fd5-be72-b7ea412058bc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:13.233149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec1485-ddbc-4b72-b837-e8b729be3594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:13.233149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1485-ddbc-4b72-b837-e8b729be3594 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.233149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1485-ddbc-4b72-b837-e8b729be3594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.234033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1485-ddbc-4b72-b837-e8b729be3594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.234633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec1485-ddbc-4b72-b837-e8b729be3594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:13.236285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec1485-ddbc-4b72-b837-e8b729be3594 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:13.281603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 756/1512] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:13.281729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:13.303186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eb7b1-4a03-4648-ba04-4d7a8dadb12e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:13.309842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:13.310575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.310856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.311126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] PATH_INFO: `/v3/projects/d3afc64e6c974e7fb68dbae209eb43ca/users/78f89f53b1e445f7b204ad2428ccdd22/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.331719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3afc64e6c974e7fb68dbae209eb43ca, user_id=78f89f53b1e445f7b204ad2428ccdd22, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:13.336788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:13.373824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 757/1513] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:13 2022] PUT /identity/v3/projects/d3afc64e6c974e7fb68dbae209eb43ca/users/78f89f53b1e445f7b204ad2428ccdd22/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:13.388385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.388811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.389260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.450600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:13.469992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] MFA Rules not processed for user `78f89f53b1e445f7b204ad2428ccdd22`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:13.471155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:13.586357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 757/1514] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:13.590466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:13.744906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967e863-a992-45af-afa6-da2e4fae2f91 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:13.753729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:13.755151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.758156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.758668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.760053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:13.763067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:13.826585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 758/1515] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:13 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:13.828781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:13.848670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b19b-5c5c-4d61-8a9e-825ce10df826 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:13.948210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:13.949347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:13.949902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:13.950406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:13.951332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:13.954476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:13.993242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 758/1516] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:13 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:13.997377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:14.015601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8189b79-a8e2-4133-a4e8-fadd2b2499ea admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:14.023791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:14.024417 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:14.024699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:14.024972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:14.025686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:14.027289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:14.040547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 759/1517] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:14.044741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:14.068915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d467587-e13d-4a29-ae0d-ba534260ad98 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:14.092490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:14.093399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:14.093520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:14.093721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] PATH_INFO: `/v3/projects/e70db7a54afc49ed8aefb1c31a3a75d3/users/af8fea239ce9474c90c5f6abe998dd0e/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:14.119326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70db7a54afc49ed8aefb1c31a3a75d3, user_id=af8fea239ce9474c90c5f6abe998dd0e, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:14.129768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:14.162420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 759/1518] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:14 2022] PUT /identity/v3/projects/e70db7a54afc49ed8aefb1c31a3a75d3/users/af8fea239ce9474c90c5f6abe998dd0e/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:14.166214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:14.177573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2650-f2bf-4273-9b62-9679c163f3ae admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:14.266668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:14.303085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59fd43-00eb-4af9-ad05-de18f33530c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:14.304629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59fd43-00eb-4af9-ad05-de18f33530c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:14.305055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59fd43-00eb-4af9-ad05-de18f33530c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:14.305469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59fd43-00eb-4af9-ad05-de18f33530c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:14.306485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59fd43-00eb-4af9-ad05-de18f33530c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:14.308406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59fd43-00eb-4af9-ad05-de18f33530c9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:14.325167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 760/1519] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:14.328920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:14.341596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258be96-8ee6-4128-a9d2-0f79f284f4f4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:14.347142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:14.347999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:14.348280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:14.348550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] PATH_INFO: `/v3/projects/e70db7a54afc49ed8aefb1c31a3a75d3/users/af8fea239ce9474c90c5f6abe998dd0e/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:14.350666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70db7a54afc49ed8aefb1c31a3a75d3, user_id=af8fea239ce9474c90c5f6abe998dd0e, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:14.353349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:14.373129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 760/1520] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:14 2022] PUT /identity/v3/projects/e70db7a54afc49ed8aefb1c31a3a75d3/users/af8fea239ce9474c90c5f6abe998dd0e/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:14.384117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:14.384399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:14.384668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:14.467301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] MFA Rules not processed for user `af8fea239ce9474c90c5f6abe998dd0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:14.468967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:14.585946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 761/1521] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:14.598587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:14.601058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:14.729078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9a62-5578-4038-a56b-2e6e52068685 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:14.732893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ba2d0-b297-481a-8e9b-def033964eb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:14.733507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ba2d0-b297-481a-8e9b-def033964eb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:14.733787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ba2d0-b297-481a-8e9b-def033964eb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:14.734054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ba2d0-b297-481a-8e9b-def033964eb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:14.744574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ba2d0-b297-481a-8e9b-def033964eb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:14.746040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ba2d0-b297-481a-8e9b-def033964eb2 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:14.750109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:14.757875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 761/1522] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:15.443044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:15.453476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530b0d1-1b16-4705-83f3-0d16987eb759 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:15.457143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:15.457774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:15.458074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:15.458337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:15.518930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:15.520581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:15.532487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 762/1523] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:18.143610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:18.144126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:18.144315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:18.169667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:18.170880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:18.243222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 762/1524] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Oct 11 06:42:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:18.277691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:18.280630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:18.320517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25759d-2b7e-4f08-8c37-063d16a114d7 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:18.333074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a040ea-c916-43c2-8e11-237e779d9b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:18.333733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a040ea-c916-43c2-8e11-237e779d9b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:18.336651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a040ea-c916-43c2-8e11-237e779d9b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:18.336651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a040ea-c916-43c2-8e11-237e779d9b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:18.355905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a040ea-c916-43c2-8e11-237e779d9b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:18.361430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a040ea-c916-43c2-8e11-237e779d9b51 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:18.384705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 763/1525] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:20.072926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:20.075228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:20.088393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3222f9-4be3-4716-8476-036ff303e365 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:20.092618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48608428-e5ee-4b48-837e-1d3a74c0ca29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:20.093226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48608428-e5ee-4b48-837e-1d3a74c0ca29 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:20.093792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48608428-e5ee-4b48-837e-1d3a74c0ca29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:20.094065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48608428-e5ee-4b48-837e-1d3a74c0ca29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:20.181668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48608428-e5ee-4b48-837e-1d3a74c0ca29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:20.184027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48608428-e5ee-4b48-837e-1d3a74c0ca29 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:20.203858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 763/1526] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:20.361541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b71a2-9e29-4d63-9833-26b3515b6530 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:20.362054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b71a2-9e29-4d63-9833-26b3515b6530 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:20.362148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b71a2-9e29-4d63-9833-26b3515b6530 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:20.364116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 764/1527] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:42:20.385962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d3e03-af7d-44b5-95f9-090f5fa33e4a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:20.386629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d3e03-af7d-44b5-95f9-090f5fa33e4a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:20.386915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d3e03-af7d-44b5-95f9-090f5fa33e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:20.452434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ca8d3e03-af7d-44b5-95f9-090f5fa33e4a None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:20.453958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ca8d3e03-af7d-44b5-95f9-090f5fa33e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:20.590527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 764/1528] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:42:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:20.654515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b71a2-9e29-4d63-9833-26b3515b6530 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:20.664138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ea0b71a2-9e29-4d63-9833-26b3515b6530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:20.704159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b71a2-9e29-4d63-9833-26b3515b6530 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:20.718750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1010-b7f6-4fcc-8489-966e3e88d613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:20.719653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1010-b7f6-4fcc-8489-966e3e88d613 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:20.720271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1010-b7f6-4fcc-8489-966e3e88d613 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:20.720850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1010-b7f6-4fcc-8489-966e3e88d613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:20.751234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1010-b7f6-4fcc-8489-966e3e88d613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:20.753612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1010-b7f6-4fcc-8489-966e3e88d613 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:20.772899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 765/1529] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:21.239152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751ca24-4be4-4e99-93ef-b3dae1aeb2b8 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:21.240548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751ca24-4be4-4e99-93ef-b3dae1aeb2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:21.241133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751ca24-4be4-4e99-93ef-b3dae1aeb2b8 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:21.243550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 765/1530] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:42:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:42:21.267602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:21.268095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:21.268667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:21.313558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:21.315180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:21.351103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 766/1531] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:42:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:21.365951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751ca24-4be4-4e99-93ef-b3dae1aeb2b8 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:21.382225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751ca24-4be4-4e99-93ef-b3dae1aeb2b8 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:21.388450 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11cb7d-9f29-4786-9412-c452efc86a40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:21.389173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cb7d-9f29-4786-9412-c452efc86a40 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:21.390125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cb7d-9f29-4786-9412-c452efc86a40 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:21.390474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cb7d-9f29-4786-9412-c452efc86a40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:21.401051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cb7d-9f29-4786-9412-c452efc86a40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:21.403568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cb7d-9f29-4786-9412-c452efc86a40 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:21.423621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 766/1532] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:24.612674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:24.655068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:24.739465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c26a54-1a3c-4160-a190-32be5503eaab None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:24.748769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:24.749638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:24.752222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:24.752740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] PATH_INFO: `/v3/users/480d50c41505423eaaf9c9653d1367e1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:24.754144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=480d50c41505423eaaf9c9653d1367e1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:24.757516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:24.883887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '480d50c41505423eaaf9c9653d1367e1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:24.884767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '480d50c41505423eaaf9c9653d1367e1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:24.899091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '480d50c41505423eaaf9c9653d1367e1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:24.900097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '480d50c41505423eaaf9c9653d1367e1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:24.919662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '480d50c41505423eaaf9c9653d1367e1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:24.960151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 767/1533] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:24 2022] DELETE /identity/v3/users/480d50c41505423eaaf9c9653d1367e1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:24.962335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:24.962335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:24.962848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:25.000027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:25.003564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:25.053486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:25.171044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 767/1534] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:25.395236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:25.398492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:25.522269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12464a64-9b9e-4ac3-9289-cbec5d41bf34 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:25.528408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:25.529070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:25.529345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:25.530087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:25.544789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:25.546605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:25.560949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 768/1535] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:26.297533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:26.299281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:26.325374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41d227-176e-49c6-99ee-8822c2e5e23a None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:26.331559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:26.332510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:26.332937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:26.333220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] PATH_INFO: `/v3/projects/fe76e86641ab4a6d95643c9a7b83debe` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:26.335107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe76e86641ab4a6d95643c9a7b83debe)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:26.337251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:26.389545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe76e86641ab4a6d95643c9a7b83debe', 'request_id': 'req-13916232-eb50-48b0-b2c6-f11087ff769f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.398903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe76e86641ab4a6d95643c9a7b83debe', 'request_id': 'req-13916232-eb50-48b0-b2c6-f11087ff769f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.409337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe76e86641ab4a6d95643c9a7b83debe', 'request_id': 'req-13916232-eb50-48b0-b2c6-f11087ff769f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.426136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] The token cache is being invalidate because project fe76e86641ab4a6d95643c9a7b83debe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:26.426480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.427390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.429168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 768/1536] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:26 2022] DELETE /identity/v3/projects/fe76e86641ab4a6d95643c9a7b83debe => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:26.433321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:26.443458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4b58b-5731-4e5f-ab5b-37c37215ed41 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:26.495949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:26.514509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:26.515209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:26.515516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:26.515793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] PATH_INFO: `/v3/users/78f89f53b1e445f7b204ad2428ccdd22` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:26.517067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f89f53b1e445f7b204ad2428ccdd22)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:26.519306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:26.574935 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f89f53b1e445f7b204ad2428ccdd22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.575630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f89f53b1e445f7b204ad2428ccdd22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.585395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f89f53b1e445f7b204ad2428ccdd22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.586095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f89f53b1e445f7b204ad2428ccdd22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.594005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f89f53b1e445f7b204ad2428ccdd22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:26.631219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 769/1537] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:26 2022] DELETE /identity/v3/users/78f89f53b1e445f7b204ad2428ccdd22 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:27.678682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:27.680206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:27.695259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916232-eb50-48b0-b2c6-f11087ff769f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:27.760502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:27.761367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:27.761668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:27.762098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] PATH_INFO: `/v3/projects/d3afc64e6c974e7fb68dbae209eb43ca` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:27.763433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3afc64e6c974e7fb68dbae209eb43ca)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:27.765566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:27.816637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3afc64e6c974e7fb68dbae209eb43ca', 'request_id': 'req-7061d199-df6e-4e6b-98e2-9854a1403454'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:27.819536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:27.830121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3afc64e6c974e7fb68dbae209eb43ca', 'request_id': 'req-7061d199-df6e-4e6b-98e2-9854a1403454'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:27.838254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3afc64e6c974e7fb68dbae209eb43ca', 'request_id': 'req-7061d199-df6e-4e6b-98e2-9854a1403454'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:27.853472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] The token cache is being invalidate because project d3afc64e6c974e7fb68dbae209eb43ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:27.853853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:27.854643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:27.856252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 769/1538] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:27 2022] DELETE /identity/v3/projects/d3afc64e6c974e7fb68dbae209eb43ca => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:27.860193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:27.861708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:27.876640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40184b-abb1-4f25-b9d8-5637f9ee390e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:27.957236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:27.957950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:27.958234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:27.958508 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] PATH_INFO: `/v3/users/af8fea239ce9474c90c5f6abe998dd0e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:27.960444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8fea239ce9474c90c5f6abe998dd0e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:27.963393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:28.045936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8fea239ce9474c90c5f6abe998dd0e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.046708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8fea239ce9474c90c5f6abe998dd0e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.047943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:28.050005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:28.058141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8fea239ce9474c90c5f6abe998dd0e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.060027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8fea239ce9474c90c5f6abe998dd0e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.073169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-fa33c810-3159-45a3-9ecb-8fd80f2c9b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8fea239ce9474c90c5f6abe998dd0e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.078298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061d199-df6e-4e6b-98e2-9854a1403454 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:28.079843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:28.093507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:28.094642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:28.095949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:28.096529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] PATH_INFO: `/v3/users/0a245d4b24ce47f4a6ba5746b57f87b7` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:28.098750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a245d4b24ce47f4a6ba5746b57f87b7)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:28.102412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:28.116808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 770/1539] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:27 2022] DELETE /identity/v3/users/af8fea239ce9474c90c5f6abe998dd0e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:28.192134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a245d4b24ce47f4a6ba5746b57f87b7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.193308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a245d4b24ce47f4a6ba5746b57f87b7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.199014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a245d4b24ce47f4a6ba5746b57f87b7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.213417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a245d4b24ce47f4a6ba5746b57f87b7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.240527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a245d4b24ce47f4a6ba5746b57f87b7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:28.273340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 770/1540] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:28 2022] DELETE /identity/v3/users/0a245d4b24ce47f4a6ba5746b57f87b7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:28.282305 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8802b-e223-4a76-9927-636ac1f4c1ba None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:28.283497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8802b-e223-4a76-9927-636ac1f4c1ba None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:28.283497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8802b-e223-4a76-9927-636ac1f4c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:28.322563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b1d8802b-e223-4a76-9927-636ac1f4c1ba None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:28.323747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b1d8802b-e223-4a76-9927-636ac1f4c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:28.417261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 771/1541] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:28.424182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:28.426257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:28.533546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4973905-b797-4bc5-b27e-5f9f509386ff admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:28.540232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba5a8a-702b-4556-bd7d-fe76389db188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:28.540868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba5a8a-702b-4556-bd7d-fe76389db188 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:28.541136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba5a8a-702b-4556-bd7d-fe76389db188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:28.541405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba5a8a-702b-4556-bd7d-fe76389db188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:28.558811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba5a8a-702b-4556-bd7d-fe76389db188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:28.562020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba5a8a-702b-4556-bd7d-fe76389db188 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:28.590217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 771/1542] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:28.845588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52296288-5059-4580-b270-2960832ae537 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:28.845932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52296288-5059-4580-b270-2960832ae537 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:28.846212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52296288-5059-4580-b270-2960832ae537 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:28.847962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 772/1543] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:42:28.881046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:28.881632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:28.881632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:28.923551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52296288-5059-4580-b270-2960832ae537 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:28.925484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-52296288-5059-4580-b270-2960832ae537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:28.939906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:28.941505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:28.951556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52296288-5059-4580-b270-2960832ae537 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:28.974502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:28.975186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:28.975487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:28.976028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] PATH_INFO: `/v3/projects/e70db7a54afc49ed8aefb1c31a3a75d3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:28.978805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70db7a54afc49ed8aefb1c31a3a75d3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:28.981292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.014444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:29.058123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 772/1544] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:42:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:29.069431 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70db7a54afc49ed8aefb1c31a3a75d3', 'request_id': 'req-976cbe5d-59aa-4114-9860-605a6261b58c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.077100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.077278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70db7a54afc49ed8aefb1c31a3a75d3', 'request_id': 'req-976cbe5d-59aa-4114-9860-605a6261b58c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.078831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.087296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70db7a54afc49ed8aefb1c31a3a75d3', 'request_id': 'req-976cbe5d-59aa-4114-9860-605a6261b58c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.099847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43339bc2-8bbf-4517-8e09-1256c49b182e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.103466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] The token cache is being invalidate because project e70db7a54afc49ed8aefb1c31a3a75d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:29.104224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.107270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-976cbe5d-59aa-4114-9860-605a6261b58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.110820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 773/1545] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:28 2022] DELETE /identity/v3/projects/e70db7a54afc49ed8aefb1c31a3a75d3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:29.116808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739096c2-c7fd-478f-92c4-23401ccb3fca None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.117097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739096c2-c7fd-478f-92c4-23401ccb3fca None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.117375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739096c2-c7fd-478f-92c4-23401ccb3fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.152512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-739096c2-c7fd-478f-92c4-23401ccb3fca None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:29.153841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-739096c2-c7fd-478f-92c4-23401ccb3fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.192965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.194679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.194998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.195320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.197539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.214267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.217191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.235590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 773/1546] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:29.238214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.241671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.274095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 774/1547] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:29.287210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.288643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.288925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.309276 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:29.310634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.319165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 775/1548] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:29.322894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.328777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:29.333534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8fd46-3f9c-45cd-b9ef-0a2a92c67d81 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.340884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.342940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.343219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.343439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f12fe-da1a-4936-b4b6-e0bf00699ace service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.343547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.344160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:29.344462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.346061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.347894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.348518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.348792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.349059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] PATH_INFO: `/v3/projects/dc40ce7a6847448790b623a616b996cd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.350769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc40ce7a6847448790b623a616b996cd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.352700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.358453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 776/1549] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:29.362516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.372080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2f34a-6a84-4c37-a685-3c495a60b32d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.376669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.377592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.377592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.377848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.378407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.380657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.411494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc40ce7a6847448790b623a616b996cd', 'request_id': 'req-0f0e159b-bb0d-4b84-8375-b43a990f9251'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.424821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc40ce7a6847448790b623a616b996cd', 'request_id': 'req-0f0e159b-bb0d-4b84-8375-b43a990f9251'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.425483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 777/1550] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:29.429569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.433138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc40ce7a6847448790b623a616b996cd', 'request_id': 'req-0f0e159b-bb0d-4b84-8375-b43a990f9251'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.441564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:29.446410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] The token cache is being invalidate because project dc40ce7a6847448790b623a616b996cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:29.446847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.448698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.450886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 774/1551] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:29 2022] DELETE /identity/v3/projects/dc40ce7a6847448790b623a616b996cd => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:29.454786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.457150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.503836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f13b79-468e-4182-b791-ec9086b7a5d2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.509039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.509655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.509934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.510204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.510859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.512913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.535493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e159b-bb0d-4b84-8375-b43a990f9251 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.541114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 778/1552] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:29.541252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.542189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.542687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.543052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] PATH_INFO: `/v3/users/5f3290a1afd940f1bb13665f2c20a8a3` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.544682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f3290a1afd940f1bb13665f2c20a8a3)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.544854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.546677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.546811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.560827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3388f5-0fef-4683-820b-af8195baa73a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.568137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.568137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.568137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.568455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.569574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.571257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.582340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 779/1553] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:29.586620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.596335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc348da-323c-419e-9b59-d86ded3c7f2c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.598434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f3290a1afd940f1bb13665f2c20a8a3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.599135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f3290a1afd940f1bb13665f2c20a8a3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.599824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f3290a1afd940f1bb13665f2c20a8a3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.601451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.602480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.602867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.603201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.603866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.605439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.607027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f3290a1afd940f1bb13665f2c20a8a3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.617749 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 780/1554] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:29.621060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-cb170c3f-c550-4c6a-8a4b-ee17ac083667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f3290a1afd940f1bb13665f2c20a8a3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:29.622364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.624206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.638012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f21694-15e3-49db-b990-969e9ab10d87 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.643606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.647851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 775/1555] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:29 2022] DELETE /identity/v3/users/5f3290a1afd940f1bb13665f2c20a8a3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:29.647985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.647985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.647985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] PATH_INFO: `/v3/projects/aeacef94e95f4e3eb8cb4974f42ce3a7/users/a248328511e54a01afa1b8477d8700c8/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.661730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeacef94e95f4e3eb8cb4974f42ce3a7, user_id=a248328511e54a01afa1b8477d8700c8, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.664506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.679625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 781/1556] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:29 2022] PUT /identity/v3/projects/aeacef94e95f4e3eb8cb4974f42ce3a7/users/a248328511e54a01afa1b8477d8700c8/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:29.691352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.691661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.691931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.730429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] MFA Rules not processed for user `a248328511e54a01afa1b8477d8700c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:29.731833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.820251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 776/1557] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:29.824748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.893359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e842bf6-7f2e-4f56-a148-8dabccdb3b11 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.898743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:29.899453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:29.900081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:29.900081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:29.900808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:29.902765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:29.919859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:29.943605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 782/1558] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:29.945609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.958578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:29.960602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e39ab-2e46-423d-857a-5f550320456f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:29.963603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:29.966558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:30.037367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.038374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.038908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.039252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.040041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.042248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.073787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 777/1559] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:29 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:30.075532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.086606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c91911-8cd0-4504-8d54-cced7a3b4ec8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.092027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.092841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.093255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.093601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.094786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.096695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.116466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 778/1560] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:30.116640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f5cf4-588f-45d8-b0af-f477413a425e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.116736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.126872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.127612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.127895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.128167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] PATH_INFO: `/v3/projects/e9c0ddd655df40ecb1a74118c6aab6cc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.129076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a16fa-3a99-4ce1-9eaa-c1a379fbdb5a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.132797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c0ddd655df40ecb1a74118c6aab6cc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.137063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.137912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.138695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.139484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.139484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] PATH_INFO: `/v3/projects/12e7de9401364248a4470fa3627dfc1d/users/03a06b47a33f492e836d1207438b0a5d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.172942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e7de9401364248a4470fa3627dfc1d, user_id=03a06b47a33f492e836d1207438b0a5d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.180849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.205455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 779/1561] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:30 2022] PUT /identity/v3/projects/12e7de9401364248a4470fa3627dfc1d/users/03a06b47a33f492e836d1207438b0a5d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:30.210417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.222314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c0ddd655df40ecb1a74118c6aab6cc', 'request_id': 'req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:30.231578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c0ddd655df40ecb1a74118c6aab6cc', 'request_id': 'req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:30.242919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c0ddd655df40ecb1a74118c6aab6cc', 'request_id': 'req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:30.259752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] The token cache is being invalidate because project e9c0ddd655df40ecb1a74118c6aab6cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:30.263588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:30.265267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b79f313b-e9d3-4574-b8ec-3e5222c0a41f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:30.268679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 783/1562] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:29 2022] DELETE /identity/v3/projects/e9c0ddd655df40ecb1a74118c6aab6cc => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:30.274426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.274553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.274743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.305505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f5eee-f23a-40f1-a4dd-461bfeeed5bc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.308544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:30.309946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:30.384318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce661a6-5d7a-4810-8794-23bd7252915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.385006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce661a6-5d7a-4810-8794-23bd7252915e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.385287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce661a6-5d7a-4810-8794-23bd7252915e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.385561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce661a6-5d7a-4810-8794-23bd7252915e admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.386254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce661a6-5d7a-4810-8794-23bd7252915e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.388122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce661a6-5d7a-4810-8794-23bd7252915e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.395790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 784/1563] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:30.402814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 780/1564] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:30.406113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.411040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:30.419072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.419356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.420136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.437864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:30.440725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72449f3f-3a18-4871-a8be-6d89af30c6e2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.445300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:30.448338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:30.451136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144646a-124e-4ab4-a0b7-fb6b52f1f425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.451843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144646a-124e-4ab4-a0b7-fb6b52f1f425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.452928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144646a-124e-4ab4-a0b7-fb6b52f1f425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.453216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144646a-124e-4ab4-a0b7-fb6b52f1f425 admin admin] PATH_INFO: `/v3/projects/12e7de9401364248a4470fa3627dfc1d/users/03a06b47a33f492e836d1207438b0a5d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.456477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144646a-124e-4ab4-a0b7-fb6b52f1f425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e7de9401364248a4470fa3627dfc1d, user_id=03a06b47a33f492e836d1207438b0a5d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.465137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144646a-124e-4ab4-a0b7-fb6b52f1f425 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.476428 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 781/1565] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:30.481306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.500010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee9a7c-6267-40b2-bd4a-cb214c812414 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.511501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 785/1566] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:30 2022] PUT /identity/v3/projects/12e7de9401364248a4470fa3627dfc1d/users/03a06b47a33f492e836d1207438b0a5d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:30.513256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.514334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.514745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.546874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:30.554294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] MFA Rules not processed for user `03a06b47a33f492e836d1207438b0a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:30.557019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:30.620120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.620662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.620937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.621205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.623691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:30.624007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.625612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.678725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 782/1567] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:30.687719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.700167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93b395-af64-43c1-94ae-07206e71256e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.705312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.706247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.706536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.706803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.711279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.711666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 786/1568] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:30.711948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.718797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.720622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:30.766189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 783/1569] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:30 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:30.771311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.842122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3645f2-b4a2-48d2-a8a7-7c8b52b0413b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.897119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9cb0-31d2-44ef-a2c9-e15f5dfcc8f5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:30.903610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.905379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.905379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.905807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.907223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:30.910660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:30.925197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:30.927036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:30.927774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:30.928878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:30.951272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 784/1570] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:30 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:30.955684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:30.997186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e51f62-1293-4655-a6ae-74927841d111 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.009614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.011035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.011929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.011929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.013792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.016166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.023627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.025249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.035546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 785/1571] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:31.039843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.043470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 787/1572] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:31.066600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edb3a3-3c9b-4fd9-af46-68036ad21233 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.080925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5f823-4640-41ab-aa41-9629eaa45d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.082652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5f823-4640-41ab-aa41-9629eaa45d54 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.083176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5f823-4640-41ab-aa41-9629eaa45d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.083509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5f823-4640-41ab-aa41-9629eaa45d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.085442 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5f823-4640-41ab-aa41-9629eaa45d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.090419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5f823-4640-41ab-aa41-9629eaa45d54 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.113035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 786/1573] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:31.117686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.125110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:31.132702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c92e6-e7b2-415b-ad0f-e68a9ec34211 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.152633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.153533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.153963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.154385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] PATH_INFO: `/v3/projects/6c25bf0ae1ef4113ab1762df415c5b8c/users/76de01b1c0c64f9c969af63ad035bfbe/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.174159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c25bf0ae1ef4113ab1762df415c5b8c, user_id=76de01b1c0c64f9c969af63ad035bfbe, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.177585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.204446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 788/1574] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:31 2022] PUT /identity/v3/projects/6c25bf0ae1ef4113ab1762df415c5b8c/users/76de01b1c0c64f9c969af63ad035bfbe/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:31.214461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.214904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.215439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.264427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] MFA Rules not processed for user `76de01b1c0c64f9c969af63ad035bfbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:31.266037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:31.322709 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:31.374197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 787/1575] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:31.378598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.452346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47a13-4b4b-4b87-bff3-ef3aee155a20 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.457160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.457853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.458210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.458600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.459260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.461724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.493740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 789/1576] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:31 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:31.494904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.505647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853ff39-410a-4fa2-a8db-5e8139046151 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.567716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.568337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.568635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.568918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.569556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.571564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.601185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 788/1577] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:31 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:31.602159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.613547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7b2ad-d948-4432-bf5c-ba1b4bdf5fa8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.619103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.619826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.620201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.620670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.621396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.623069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.635496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 790/1578] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:31.639978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.650824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783005bf-aeb2-43cf-94b5-916569d10357 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.655723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.656375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.656651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.656912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] PATH_INFO: `/v3/projects/b108a0177ab44d41a1d936687982d6cf/users/26028aa1e6214b8ca9b9ff3d7d7b62d1/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.673602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b108a0177ab44d41a1d936687982d6cf, user_id=26028aa1e6214b8ca9b9ff3d7d7b62d1, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.676301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.690482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 789/1579] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:31 2022] PUT /identity/v3/projects/b108a0177ab44d41a1d936687982d6cf/users/26028aa1e6214b8ca9b9ff3d7d7b62d1/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:31.695561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.707044 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98dc7-e8a8-4972-ac88-f1f697fec82c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.772943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.785835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.786451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.786729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.787002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.788229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.790032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.799945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 791/1580] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:31.804108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.813975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb181b85-c1f6-474c-9727-ef8e72d8918d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.819163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35baeec-0f5e-44ef-b641-0c97eb91919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.819847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35baeec-0f5e-44ef-b641-0c97eb91919d admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.820129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35baeec-0f5e-44ef-b641-0c97eb91919d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.820400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35baeec-0f5e-44ef-b641-0c97eb91919d admin admin] PATH_INFO: `/v3/projects/b108a0177ab44d41a1d936687982d6cf/users/26028aa1e6214b8ca9b9ff3d7d7b62d1/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.822362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35baeec-0f5e-44ef-b641-0c97eb91919d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b108a0177ab44d41a1d936687982d6cf, user_id=26028aa1e6214b8ca9b9ff3d7d7b62d1, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.825307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35baeec-0f5e-44ef-b641-0c97eb91919d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.834294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7335dd0-ec92-442c-a567-40122ea0cba1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:31.835885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 792/1581] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:31 2022] PUT /identity/v3/projects/b108a0177ab44d41a1d936687982d6cf/users/26028aa1e6214b8ca9b9ff3d7d7b62d1/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:31.847171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.847591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.847731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.879134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] MFA Rules not processed for user `26028aa1e6214b8ca9b9ff3d7d7b62d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:31.880404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:31.889850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:31.911019 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:31.911182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:31.911625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:31.911923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:31.912256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] PATH_INFO: `/v3/users/a248328511e54a01afa1b8477d8700c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:31.913540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a248328511e54a01afa1b8477d8700c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:31.915626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:31.969046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 793/1582] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:31.973316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a248328511e54a01afa1b8477d8700c8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:31.975056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a248328511e54a01afa1b8477d8700c8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:31.975681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a248328511e54a01afa1b8477d8700c8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:31.981337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:31.983289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a248328511e54a01afa1b8477d8700c8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:31.983498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:31.992853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-26fa2540-7719-4800-b221-e62c0ff6dfaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a248328511e54a01afa1b8477d8700c8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:32.011016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 790/1583] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:31 2022] DELETE /identity/v3/users/a248328511e54a01afa1b8477d8700c8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:32.015438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:32.015782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:32.016057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:32.035740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:32.036952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:32.046582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6320df-93a8-4f6f-b9f0-6a4c1902ff9b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:32.096315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:32.096924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:32.097197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:32.097511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:32.099237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:32.102913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 791/1584] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:32.113021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:32.115026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:32.190915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:32.192792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:32.202047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf548f-1954-4c49-a1ee-8113dce95879 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:32.204622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 794/1585] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:32.213110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e3f04-ac71-4e2f-b7c5-e44a3670072c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:32.214175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e3f04-ac71-4e2f-b7c5-e44a3670072c service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:32.214672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e3f04-ac71-4e2f-b7c5-e44a3670072c service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:32.215312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e3f04-ac71-4e2f-b7c5-e44a3670072c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:32.226562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e3f04-ac71-4e2f-b7c5-e44a3670072c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:32.228902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e3f04-ac71-4e2f-b7c5-e44a3670072c service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:32.248758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 792/1586] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:32.280303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:32.282283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:32.301416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a30d-56fc-4ef2-bd3d-ea1ffe8fe7a6 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:32.308199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:32.309049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:32.309397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:32.310360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] PATH_INFO: `/v3/users/76de01b1c0c64f9c969af63ad035bfbe` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:32.311895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76de01b1c0c64f9c969af63ad035bfbe)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:32.314733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:32.345195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:32.381374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76de01b1c0c64f9c969af63ad035bfbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:32.382238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76de01b1c0c64f9c969af63ad035bfbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:32.391375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76de01b1c0c64f9c969af63ad035bfbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:32.392183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76de01b1c0c64f9c969af63ad035bfbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:32.413673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76de01b1c0c64f9c969af63ad035bfbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:32.453752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 795/1587] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:32 2022] DELETE /identity/v3/users/76de01b1c0c64f9c969af63ad035bfbe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:32.458459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:32.458690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:32.458978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:32.468287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:32.483671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:32.485147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:32.627658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 793/1588] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:32.663776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:32.665267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:32.780820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ffae-2c2f-451c-93cf-9397113d9fb6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:32.786285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:32.787227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:32.788262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:32.788727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:32.802168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:32.805704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:32.834072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 796/1589] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:32.984602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:32.986154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:33.002473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd331de-b41d-4e62-ada2-a7d2b8dcb146 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:33.010041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:33.010700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:33.010974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:33.011240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] PATH_INFO: `/v3/projects/aeacef94e95f4e3eb8cb4974f42ce3a7` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:33.012750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeacef94e95f4e3eb8cb4974f42ce3a7)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:33.014853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:33.104116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeacef94e95f4e3eb8cb4974f42ce3a7', 'request_id': 'req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.112932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeacef94e95f4e3eb8cb4974f42ce3a7', 'request_id': 'req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.124353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeacef94e95f4e3eb8cb4974f42ce3a7', 'request_id': 'req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.140311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] The token cache is being invalidate because project aeacef94e95f4e3eb8cb4974f42ce3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:33.140670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.141310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.144145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 794/1590] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:32 2022] DELETE /identity/v3/projects/aeacef94e95f4e3eb8cb4974f42ce3a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:33.148894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:33.169815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fcb4f-c906-4220-8b14-d2a658f890fd service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:33.244103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:33.244748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:33.245024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:33.245300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] PATH_INFO: `/v3/users/03a06b47a33f492e836d1207438b0a5d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:33.249238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:33.250275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a06b47a33f492e836d1207438b0a5d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:33.251682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:33.252634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:33.269271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7633f6-4ffa-4c36-a9eb-9c9058a548c7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:33.278223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:33.283500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:33.283972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:33.284269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] PATH_INFO: `/v3/projects/6c25bf0ae1ef4113ab1762df415c5b8c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:33.285996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c25bf0ae1ef4113ab1762df415c5b8c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:33.289406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:33.321033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:33.322866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a06b47a33f492e836d1207438b0a5d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.323681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a06b47a33f492e836d1207438b0a5d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.335402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a06b47a33f492e836d1207438b0a5d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.336229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a06b47a33f492e836d1207438b0a5d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.344264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a06b47a33f492e836d1207438b0a5d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.349653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c25bf0ae1ef4113ab1762df415c5b8c', 'request_id': 'req-6f824925-ad94-4d13-9079-9b927f9fc921'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.358957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c25bf0ae1ef4113ab1762df415c5b8c', 'request_id': 'req-6f824925-ad94-4d13-9079-9b927f9fc921'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.364732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 797/1591] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:33 2022] DELETE /identity/v3/users/03a06b47a33f492e836d1207438b0a5d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:33.367822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c25bf0ae1ef4113ab1762df415c5b8c', 'request_id': 'req-6f824925-ad94-4d13-9079-9b927f9fc921'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.379850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] The token cache is being invalidate because project 6c25bf0ae1ef4113ab1762df415c5b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:33.380268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.380975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.383557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 795/1592] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:33 2022] DELETE /identity/v3/projects/6c25bf0ae1ef4113ab1762df415c5b8c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:33.387798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:33.389788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:33.556211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092620b-8578-4b7d-aaa3-d162aca01c65 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:33.574906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:33.575984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:33.576477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:33.577061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] PATH_INFO: `/v3/users/26028aa1e6214b8ca9b9ff3d7d7b62d1` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:33.580250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26028aa1e6214b8ca9b9ff3d7d7b62d1)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:33.582332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:33.666569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26028aa1e6214b8ca9b9ff3d7d7b62d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.667641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26028aa1e6214b8ca9b9ff3d7d7b62d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.680279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26028aa1e6214b8ca9b9ff3d7d7b62d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.681167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26028aa1e6214b8ca9b9ff3d7d7b62d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.695948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26028aa1e6214b8ca9b9ff3d7d7b62d1'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:33.735888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 798/1593] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:33 2022] DELETE /identity/v3/users/26028aa1e6214b8ca9b9ff3d7d7b62d1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:34.031530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.033087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:34.056703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:34.130972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824925-ad94-4d13-9079-9b927f9fc921 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.211920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.212861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.213316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.213779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] PATH_INFO: `/v3/projects/12e7de9401364248a4470fa3627dfc1d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.215477 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e7de9401364248a4470fa3627dfc1d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.221513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.255003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.257073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:34.270441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697618-e4ec-451c-93ba-da84ba233f10 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.285792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e7de9401364248a4470fa3627dfc1d', 'request_id': 'req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.308058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e7de9401364248a4470fa3627dfc1d', 'request_id': 'req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.320468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e7de9401364248a4470fa3627dfc1d', 'request_id': 'req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.339252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:34.357184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] The token cache is being invalidate because project 12e7de9401364248a4470fa3627dfc1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:34.358668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.359720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b5ed2114-e0ee-45ff-bd62-3010a7d52bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.361145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 796/1594] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:34 2022] DELETE /identity/v3/projects/12e7de9401364248a4470fa3627dfc1d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:34.366340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c730f0-8a5b-4211-9d45-f91520a0ec19 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.366881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c730f0-8a5b-4211-9d45-f91520a0ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.367230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c730f0-8a5b-4211-9d45-f91520a0ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.373996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.374988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.375441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.375948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] PATH_INFO: `/v3/projects/b108a0177ab44d41a1d936687982d6cf` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.378551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b108a0177ab44d41a1d936687982d6cf)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.380883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.394590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b0c730f0-8a5b-4211-9d45-f91520a0ec19 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:34.397391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b0c730f0-8a5b-4211-9d45-f91520a0ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:34.426394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 797/1595] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:34.437295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.437636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.437914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.461609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b108a0177ab44d41a1d936687982d6cf', 'request_id': 'req-091810ac-0eff-4394-b454-966293aec060'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.464799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:34.465988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:34.468555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b108a0177ab44d41a1d936687982d6cf', 'request_id': 'req-091810ac-0eff-4394-b454-966293aec060'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.480043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b108a0177ab44d41a1d936687982d6cf', 'request_id': 'req-091810ac-0eff-4394-b454-966293aec060'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.493371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] The token cache is being invalidate because project b108a0177ab44d41a1d936687982d6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:34.493660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.494368 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-091810ac-0eff-4394-b454-966293aec060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:34.496911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 799/1596] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:34 2022] DELETE /identity/v3/projects/b108a0177ab44d41a1d936687982d6cf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:34.500903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51126070-2141-4d82-bf3c-e548a9a632bf None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.501315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51126070-2141-4d82-bf3c-e548a9a632bf None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.501599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51126070-2141-4d82-bf3c-e548a9a632bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.515686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:34.525632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-51126070-2141-4d82-bf3c-e548a9a632bf None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:34.527439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-51126070-2141-4d82-bf3c-e548a9a632bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:34.543239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 798/1597] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:34.547406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.558315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187a9d6-17b6-4f62-82fa-a5b3a4e6878e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.566570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.568073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.569843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.569843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.569843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:34.570225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.571840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.589678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 799/1598] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:34.591352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.600846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 800/1599] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:34.602794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c6789-60cb-4c53-b2d9-45091dcb13a2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.607815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.608216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.608422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.608534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.608699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.608794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.608972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.609669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.611578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.630196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:34.632080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:34.640399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 801/1600] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:34.642768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 800/1601] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:34.644729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.646818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.655311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be669fbc-64e4-4dc8-adee-c9c634c13b4f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.715359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b516-f3b5-4387-a897-3cf427bc2e05 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.720403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.721023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.721304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.721579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.722222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.723067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.723785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.724063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.724220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.724313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.724993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:34.725289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.726915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.736810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:34.745973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 802/1602] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:34.747399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.752055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 801/1603] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:34.756839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.761706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4379b8-12db-492f-be82-79b0d05c6070 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.766644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.767614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.768064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe1a9-d940-4ec8-8f27-5b5fa932504a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.768474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.769210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.770373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.772827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.773315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.773425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.773705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.773973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.774618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.776271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.787904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 802/1604] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:34.791742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.803891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641ce8-5376-4eaa-8c28-c6ded82acf29 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.811336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 803/1605] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:34.811541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.811541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.811541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.811541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.811899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.811899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.813405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.827161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 803/1606] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:34.827907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.839643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014125af-49bd-467c-b5f5-a0951a9484e2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.900031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1f1c-f63d-45db-8679-daa55f11eebb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.905088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.905696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.905975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.906374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.907018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.908968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.909073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07d20-045c-4b52-9082-e1c568b49300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:34.909724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07d20-045c-4b52-9082-e1c568b49300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.910006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07d20-045c-4b52-9082-e1c568b49300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.910280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07d20-045c-4b52-9082-e1c568b49300 admin admin] PATH_INFO: `/v3/projects/f17a8875b619436283c905e5ea9f4eb4/users/6a2c1f7106f44d63af84c222488f530d/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:34.930768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c07d20-045c-4b52-9082-e1c568b49300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17a8875b619436283c905e5ea9f4eb4, user_id=6a2c1f7106f44d63af84c222488f530d, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:34.933834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c07d20-045c-4b52-9082-e1c568b49300 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:34.938145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 804/1607] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:34.940021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:34.946283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:34.952985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b073c-330f-4cd8-9526-47e3d04d7c3d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:34.955071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 804/1608] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:34 2022] PUT /identity/v3/projects/f17a8875b619436283c905e5ea9f4eb4/users/6a2c1f7106f44d63af84c222488f530d/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:34.966118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:34.966757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:34.967261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.020145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] MFA Rules not processed for user `6a2c1f7106f44d63af84c222488f530d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:35.021684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:35.029220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.030392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.030665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.030979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.031676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.034137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.050650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 805/1609] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:35.051649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.070041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f57f3-9545-4806-a49c-c42819b38693 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.076868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.077510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.077836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.078111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.078770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.080711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.095506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 806/1610] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:35.100644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.113184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7d70a-6d30-497e-a754-25042f2fec4f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.120139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 805/1611] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:35.122984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489eb2bf-3e5c-4525-a5dc-a7d9e063b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.123942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489eb2bf-3e5c-4525-a5dc-a7d9e063b919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.124091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.124221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489eb2bf-3e5c-4525-a5dc-a7d9e063b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.124498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489eb2bf-3e5c-4525-a5dc-a7d9e063b919 admin admin] PATH_INFO: `/v3/projects/65b9cbf797a44f858facd48e9f60f791/users/24ec9b8667f04eb18e5d314cd39b1018/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.137717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb63785-414b-4afb-90e7-e5d335f2d3cf None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.143042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.143295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489eb2bf-3e5c-4525-a5dc-a7d9e063b919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b9cbf797a44f858facd48e9f60f791, user_id=24ec9b8667f04eb18e5d314cd39b1018, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.143943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.144224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.144496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.145144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.146219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489eb2bf-3e5c-4525-a5dc-a7d9e063b919 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.147661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.164380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 807/1612] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:35 2022] PUT /identity/v3/projects/65b9cbf797a44f858facd48e9f60f791/users/24ec9b8667f04eb18e5d314cd39b1018/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:35.174410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.174696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.174967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.203945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 806/1613] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:35.211473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] MFA Rules not processed for user `24ec9b8667f04eb18e5d314cd39b1018`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:35.212539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.213076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:35.238022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:35.304711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9045ee-8a3e-4747-b41b-d6d8a25b209b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.311141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.311798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.312080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.312511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 808/1614] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:35.312967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.313698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.316125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.316925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.328291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc250c9-c1ec-457f-8041-2e36dfb942b9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.334828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.335717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.336045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.336327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.337117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.339497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.347879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 807/1615] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:35.354363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.370220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994cf2a-52d9-4438-a2f3-91afbae73f26 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.375205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.375856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.376490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.376611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 809/1616] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:35.376903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.377425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.378040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.379165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.393305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 808/1617] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:35.397306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.419034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240b780-e8e3-42c7-a17e-356b802338f5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.451160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83391b0-9ff0-4f6a-9327-14e4208b4d6b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.457056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.458037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.458322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.460312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.461197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.462993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.470229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:35.488519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.489530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.489874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.490273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.491160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.492837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.493507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 810/1618] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/users => generated 302 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:35.497318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.508982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41a6d4-3e6d-4323-937a-8a562a66aecd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.512393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 809/1619] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:35.514821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.515773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.516055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.516303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.516958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.519240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.526389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.531835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 811/1620] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:35.536648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.544046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10495c79-f85d-4c16-aac4-8502c166fc8d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.548840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6969b8-6a3b-48df-9666-8e547ee937f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.549625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6969b8-6a3b-48df-9666-8e547ee937f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.549910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6969b8-6a3b-48df-9666-8e547ee937f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.550182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6969b8-6a3b-48df-9666-8e547ee937f1 admin admin] PATH_INFO: `/v3/projects/744f9390bb16492da72fc674e3ffc344/users/d2e96eab0bca4ca59daed67e9d0a8b4c/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.550419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70a41-0304-4fc4-a454-0882011d1b2b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.556438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.557383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.557865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.558340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] PATH_INFO: `/v3/projects/f51ec5890950444a9ff1d5385659df04/users/9cf65f8d36b040959b545095775fbe02/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.566713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6969b8-6a3b-48df-9666-8e547ee937f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744f9390bb16492da72fc674e3ffc344, user_id=d2e96eab0bca4ca59daed67e9d0a8b4c, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.570234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6969b8-6a3b-48df-9666-8e547ee937f1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.581974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51ec5890950444a9ff1d5385659df04, user_id=9cf65f8d36b040959b545095775fbe02, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.586654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 810/1621] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:35 2022] PUT /identity/v3/projects/744f9390bb16492da72fc674e3ffc344/users/d2e96eab0bca4ca59daed67e9d0a8b4c/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:35.587724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.598376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.598642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.599098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.612356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 812/1622] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:35 2022] PUT /identity/v3/projects/f51ec5890950444a9ff1d5385659df04/users/9cf65f8d36b040959b545095775fbe02/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:35.617396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.645330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] MFA Rules not processed for user `d2e96eab0bca4ca59daed67e9d0a8b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:35.648113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:35.721731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:35.736518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9f9b-8b3b-4a51-b32d-817638a7e8a4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.741297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.741840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.742115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.742381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.743019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.744700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.758790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 813/1623] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:35.758967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 811/1624] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:35.760859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.760992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.771547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd5b4d-80e0-474c-8082-70f128b78823 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.772381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622036b-05ff-44a7-8801-4fa52cef72f1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.776899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.777587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.777754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72320e5d-b1cc-47a8-81f3-f315dc3147d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.777880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.778028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.778662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.778766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72320e5d-b1cc-47a8-81f3-f315dc3147d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.779105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72320e5d-b1cc-47a8-81f3-f315dc3147d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.779536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72320e5d-b1cc-47a8-81f3-f315dc3147d9 admin admin] PATH_INFO: `/v3/projects/f51ec5890950444a9ff1d5385659df04/users/9cf65f8d36b040959b545095775fbe02/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.780861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.782073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72320e5d-b1cc-47a8-81f3-f315dc3147d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51ec5890950444a9ff1d5385659df04, user_id=9cf65f8d36b040959b545095775fbe02, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.785013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72320e5d-b1cc-47a8-81f3-f315dc3147d9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.797538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 814/1625] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:35 2022] PUT /identity/v3/projects/f51ec5890950444a9ff1d5385659df04/users/9cf65f8d36b040959b545095775fbe02/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:35.808879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.809349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.809619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.822122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 812/1626] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:35.826202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.844436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] MFA Rules not processed for user `9cf65f8d36b040959b545095775fbe02`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:35.846635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:35.895940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22516067-0f4f-438a-a12f-97c8358dc5e6 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.901407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.902143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.902531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.902935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.903737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.906098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.932358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 813/1627] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:35.932550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 815/1628] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:35 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:35.936859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.944901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.947407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:35.948587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807312-349a-4dfa-b0ae-154793c6bd29 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.954432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.955181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.955563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.955922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:35.956802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:35.958527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:35.969539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 814/1629] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:35.973514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:35.978952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:35.987448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaec94-51a0-4ead-8d5c-cf9878febbc2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:35.993785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:35.994455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:35.996219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:35.996518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] PATH_INFO: `/v3/projects/37860682088b49299e718f51c03daa8e/users/d476e5274902482ea43c49245d666ac3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.014004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37860682088b49299e718f51c03daa8e, user_id=d476e5274902482ea43c49245d666ac3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:36.016816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:36.023560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320ecc-9ce0-4f7d-9eaf-567f8d1318c4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:36.034083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 815/1630] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:35 2022] PUT /identity/v3/projects/37860682088b49299e718f51c03daa8e/users/d476e5274902482ea43c49245d666ac3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:36.038045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:36.081492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:36.082360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:36.082784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:36.083650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.109669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b078501-8333-4ec2-8843-208d65aa5e92 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:36.114860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:36.116192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:36.116608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:36.117229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.118020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:36.119780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:36.130782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 816/1631] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:36.132628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:36.135807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:36.152309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62204dfd-f5e2-4118-bedb-65f445d425e7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:36.160229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bc56a-04f3-4e1c-ae03-25c2e06b9f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:36.161870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bc56a-04f3-4e1c-ae03-25c2e06b9f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:36.162372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bc56a-04f3-4e1c-ae03-25c2e06b9f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:36.162831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bc56a-04f3-4e1c-ae03-25c2e06b9f92 admin admin] PATH_INFO: `/v3/projects/37860682088b49299e718f51c03daa8e/users/d476e5274902482ea43c49245d666ac3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.165230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bc56a-04f3-4e1c-ae03-25c2e06b9f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37860682088b49299e718f51c03daa8e, user_id=d476e5274902482ea43c49245d666ac3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:36.169733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0bc56a-04f3-4e1c-ae03-25c2e06b9f92 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:36.171480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:36.173024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:36.187174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 817/1632] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:36 2022] PUT /identity/v3/projects/37860682088b49299e718f51c03daa8e/users/d476e5274902482ea43c49245d666ac3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:36.187964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 816/1633] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:35 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:36.198519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:36.199023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:36.199543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.252454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] MFA Rules not processed for user `d476e5274902482ea43c49245d666ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:36.262981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:36.271786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:36.274633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:36.286749 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8555c78-621b-4f3a-9708-2220943a05d7 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:36.437893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 818/1634] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:36.447070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:36.526002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:36.541824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:36.543738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:36.544132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.574218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:36.585260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c8977-5fc9-4f92-a10b-8b3bfcaf2d71 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:36.596637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:36.597279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:36.597561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:36.597837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:36.656904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:36.658549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:36.669594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 817/1635] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:36.721817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:36.723291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:36.749663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 819/1636] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:40.977112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:40.988179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b32d0-fbc6-4b2a-8330-e991c3c85cee service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:41.000934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:41.001874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:41.002721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:41.003056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:41.142865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:41.146298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:41.176064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 818/1637] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:41.891327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:41.901286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1518f38-6ba7-4d84-9b27-d8c6a98c1c51 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:41.905505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403939b1-49a6-4ab0-8206-578c92f93f7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:41.906611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403939b1-49a6-4ab0-8206-578c92f93f7b service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:41.906611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403939b1-49a6-4ab0-8206-578c92f93f7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:41.906851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403939b1-49a6-4ab0-8206-578c92f93f7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:41.916435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403939b1-49a6-4ab0-8206-578c92f93f7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:41.917963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403939b1-49a6-4ab0-8206-578c92f93f7b service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:41.928817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 820/1638] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:42.861014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:42.881670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:42.933986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55780dbc-74d6-4a69-bbca-faaf90dcde95 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:42.939200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:42.939810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:42.940085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:42.940356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] PATH_INFO: `/v3/users/24ec9b8667f04eb18e5d314cd39b1018` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:42.941544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ec9b8667f04eb18e5d314cd39b1018)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:42.944149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:42.997206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ec9b8667f04eb18e5d314cd39b1018'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:42.997990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ec9b8667f04eb18e5d314cd39b1018'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:43.007170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ec9b8667f04eb18e5d314cd39b1018'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:43.007897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ec9b8667f04eb18e5d314cd39b1018'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:43.016599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ec9b8667f04eb18e5d314cd39b1018'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:43.036650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 819/1639] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:42 2022] DELETE /identity/v3/users/24ec9b8667f04eb18e5d314cd39b1018 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:43.041538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:43.041996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:43.042084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:43.084683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:43.088809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:43.171076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 821/1640] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:43.184621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:43.186356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:43.273270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd67ae8-9539-4445-a116-a4b6e7509f71 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:43.277750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:43.278482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:43.278764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:43.279038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:43.291402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:43.293086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:43.304982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 820/1641] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:44.293297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:44.294874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:44.306881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb831a-48bd-45a0-bdd9-6fa22bb05469 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:44.311307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:44.311959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:44.312234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:44.312499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] PATH_INFO: `/v3/projects/65b9cbf797a44f858facd48e9f60f791` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:44.313603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b9cbf797a44f858facd48e9f60f791)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:44.315526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:44.374332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b9cbf797a44f858facd48e9f60f791', 'request_id': 'req-7ca258d0-52b0-4213-856d-3af449c536c0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.397232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b9cbf797a44f858facd48e9f60f791', 'request_id': 'req-7ca258d0-52b0-4213-856d-3af449c536c0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.405690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b9cbf797a44f858facd48e9f60f791', 'request_id': 'req-7ca258d0-52b0-4213-856d-3af449c536c0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.417453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] The token cache is being invalidate because project 65b9cbf797a44f858facd48e9f60f791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:44.417888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.419406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.421566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 822/1642] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:44 2022] DELETE /identity/v3/projects/65b9cbf797a44f858facd48e9f60f791 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:44.425669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:44.435435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ece65-a600-4a78-9af4-3bd0c3ace58c service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:44.476426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:44.497782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:44.498563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:44.498900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:44.499561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] PATH_INFO: `/v3/users/9cf65f8d36b040959b545095775fbe02` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:44.500451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf65f8d36b040959b545095775fbe02)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:44.502586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:44.557860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf65f8d36b040959b545095775fbe02'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.558905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf65f8d36b040959b545095775fbe02'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.567131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf65f8d36b040959b545095775fbe02'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.567822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf65f8d36b040959b545095775fbe02'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.579942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-735fd2d1-4a0a-4519-b0a8-883199007be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf65f8d36b040959b545095775fbe02'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:44.604089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 821/1643] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:44 2022] DELETE /identity/v3/users/9cf65f8d36b040959b545095775fbe02 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:45.392517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:45.394202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:45.425324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca258d0-52b0-4213-856d-3af449c536c0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:45.434496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:45.509482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:45.510340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:45.510762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:45.511295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] PATH_INFO: `/v3/projects/f51ec5890950444a9ff1d5385659df04` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:45.513120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f51ec5890950444a9ff1d5385659df04)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:45.515241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:45.584483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f51ec5890950444a9ff1d5385659df04', 'request_id': 'req-dc791e1a-bfe7-4813-aaaf-23a57f945b83'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:45.594899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f51ec5890950444a9ff1d5385659df04', 'request_id': 'req-dc791e1a-bfe7-4813-aaaf-23a57f945b83'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:45.613078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f51ec5890950444a9ff1d5385659df04', 'request_id': 'req-dc791e1a-bfe7-4813-aaaf-23a57f945b83'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:45.630526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] The token cache is being invalidate because project f51ec5890950444a9ff1d5385659df04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:45.630847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:45.631599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dc791e1a-bfe7-4813-aaaf-23a57f945b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:45.633182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 823/1644] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:45 2022] DELETE /identity/v3/projects/f51ec5890950444a9ff1d5385659df04 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:45.638080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:45.638613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:45.638783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:45.659086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:45.660337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:45.730914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 822/1645] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:45.738981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:45.739270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:45.739630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:45.759387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:45.760828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:45.771080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 824/1646] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:45.775977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:45.786538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2d7f6-3aff-49e5-9b39-7d4abb974550 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:45.797691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:45.798387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:45.798843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:45.799290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:45.800010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:45.803192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:45.804770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:45.820005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 823/1647] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:42:45.824980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:45.834886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b5493-bbe4-4f36-828f-f1ab31dd599b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:45.840268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:45.841259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:45.841541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:45.841834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:45.842484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:45.844480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:45.871869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 825/1648] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:45 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:45.876248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:45.886005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a14e0-f10b-462d-a53a-1e4e168813a9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:45.932514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:45.947089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:45.947906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:45.948320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:45.948690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:45.949452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:45.951522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:45.979871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 824/1649] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:45 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:45.981164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:45.992614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689522d-3ed4-4389-bcb1-107a79668d8d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:46.000606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:46.000606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.000606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.000606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.000956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:46.001245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:46.015021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 826/1650] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:46.016563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:46.027450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d8328-d1e0-48a8-ade0-ea313955eb8b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:46.032728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441cb3b-f17a-45e8-afbe-3ebfaf40fde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:46.033452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441cb3b-f17a-45e8-afbe-3ebfaf40fde5 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.033796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441cb3b-f17a-45e8-afbe-3ebfaf40fde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.034156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441cb3b-f17a-45e8-afbe-3ebfaf40fde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.034818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441cb3b-f17a-45e8-afbe-3ebfaf40fde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:46.037493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441cb3b-f17a-45e8-afbe-3ebfaf40fde5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:46.053150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 825/1651] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:42:46.054454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:46.066060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce65775-370d-454b-b470-c7077ef51c14 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:46.071795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:46.072635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.073033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.073541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] PATH_INFO: `/v3/projects/991e976345ea4a988bbe706fdd27766e/users/d464c945bf954016b8f7291b659cabbe/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.092550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991e976345ea4a988bbe706fdd27766e, user_id=d464c945bf954016b8f7291b659cabbe, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:46.095632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:46.127555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 827/1652] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:46 2022] PUT /identity/v3/projects/991e976345ea4a988bbe706fdd27766e/users/d464c945bf954016b8f7291b659cabbe/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:46.129505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.129882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.130456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.184100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] MFA Rules not processed for user `d464c945bf954016b8f7291b659cabbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:46.185283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:46.238258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:46.240827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:46.300000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 826/1653] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:46.316285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:46.322545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:46.354834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:46.394743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f850d-3e0b-4466-bc18-9d77f5efea49 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:46.402181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:46.402924 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.403223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.403545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] PATH_INFO: `/v3/users/6a2c1f7106f44d63af84c222488f530d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.405000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2c1f7106f44d63af84c222488f530d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:46.407825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:46.451615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbe492-4784-4c2b-aa80-a177a171da0b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:46.460680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7af09-9b3f-4f87-a135-0347bea0dc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:46.461737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7af09-9b3f-4f87-a135-0347bea0dc26 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.462215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7af09-9b3f-4f87-a135-0347bea0dc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.462630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7af09-9b3f-4f87-a135-0347bea0dc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.476243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2c1f7106f44d63af84c222488f530d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:46.476999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2c1f7106f44d63af84c222488f530d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:46.477628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2c1f7106f44d63af84c222488f530d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:46.478442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7af09-9b3f-4f87-a135-0347bea0dc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:46.482695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7af09-9b3f-4f87-a135-0347bea0dc26 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:46.492509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2c1f7106f44d63af84c222488f530d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:46.502006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2c1f7106f44d63af84c222488f530d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:46.505121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 827/1654] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:42:46.546441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 828/1655] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:46 2022] DELETE /identity/v3/users/6a2c1f7106f44d63af84c222488f530d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:46.547998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.548297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.548568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.575871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:46.577433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:46.650131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:46.670278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 828/1656] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:46.680612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:46.683337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:46.768996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec51f-45d1-44f1-b4da-5cf14b1dc8a8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:46.773349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:46.773973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:46.774256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:46.774531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:46.786151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:46.788722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:46.803046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 829/1657] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:47.266563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:47.268792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:47.294129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1459ff-14ed-4abf-9383-e77bf0329ced None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:47.321266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:47.323194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:47.323526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:47.325792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] PATH_INFO: `/v3/projects/f17a8875b619436283c905e5ea9f4eb4` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:47.328655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17a8875b619436283c905e5ea9f4eb4)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:47.330757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:47.444086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17a8875b619436283c905e5ea9f4eb4', 'request_id': 'req-c70039bd-a5ae-458c-a603-b8cbf131f4ac'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.457953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17a8875b619436283c905e5ea9f4eb4', 'request_id': 'req-c70039bd-a5ae-458c-a603-b8cbf131f4ac'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.473604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17a8875b619436283c905e5ea9f4eb4', 'request_id': 'req-c70039bd-a5ae-458c-a603-b8cbf131f4ac'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.486535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] The token cache is being invalidate because project f17a8875b619436283c905e5ea9f4eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:47.486899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.487751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.489571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 829/1658] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:47 2022] DELETE /identity/v3/projects/f17a8875b619436283c905e5ea9f4eb4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:47.494254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:47.505964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d5882-37dd-4543-b636-bd10b440007d service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:47.573972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:47.575271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:47.575706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:47.576065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] PATH_INFO: `/v3/users/d2e96eab0bca4ca59daed67e9d0a8b4c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:47.578459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e96eab0bca4ca59daed67e9d0a8b4c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:47.580814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:47.582208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:47.635177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e96eab0bca4ca59daed67e9d0a8b4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.636011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e96eab0bca4ca59daed67e9d0a8b4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.636616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e96eab0bca4ca59daed67e9d0a8b4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.644572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e96eab0bca4ca59daed67e9d0a8b4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.653123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e96eab0bca4ca59daed67e9d0a8b4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:47.672933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 830/1659] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:47 2022] DELETE /identity/v3/users/d2e96eab0bca4ca59daed67e9d0a8b4c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:48.137019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:48.138546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:48.152326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70039bd-a5ae-458c-a603-b8cbf131f4ac admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:48.218564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:48.219242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:48.219667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:48.219947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] PATH_INFO: `/v3/projects/744f9390bb16492da72fc674e3ffc344` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:48.221107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744f9390bb16492da72fc674e3ffc344)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:48.223131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:48.271032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744f9390bb16492da72fc674e3ffc344', 'request_id': 'req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.277318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744f9390bb16492da72fc674e3ffc344', 'request_id': 'req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.284934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744f9390bb16492da72fc674e3ffc344', 'request_id': 'req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.297241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] The token cache is being invalidate because project 744f9390bb16492da72fc674e3ffc344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:48.297551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.298168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.299664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 830/1660] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:48 2022] DELETE /identity/v3/projects/744f9390bb16492da72fc674e3ffc344 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:48.304157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:48.305761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:48.318312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5363d70-423f-49c9-a18c-9e1a4005a3ca admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:48.374729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:48.375361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:48.375673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:48.375947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] PATH_INFO: `/v3/users/d476e5274902482ea43c49245d666ac3` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:48.377072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d476e5274902482ea43c49245d666ac3)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:48.378973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:48.429604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd476e5274902482ea43c49245d666ac3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.430257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd476e5274902482ea43c49245d666ac3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.430863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd476e5274902482ea43c49245d666ac3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.438050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd476e5274902482ea43c49245d666ac3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.446450 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-63997dd3-5f10-41ed-8c05-5e3c9ab4019b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd476e5274902482ea43c49245d666ac3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:48.461677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:48.469615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 831/1661] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:48 2022] DELETE /identity/v3/users/d476e5274902482ea43c49245d666ac3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:48.899738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:48.901353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:48.916816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf7181-3465-4a0f-8e76-8a34f832c8c6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:48.926469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:48.978847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:48.979595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:48.979874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:48.980144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] PATH_INFO: `/v3/projects/37860682088b49299e718f51c03daa8e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:48.981304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37860682088b49299e718f51c03daa8e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:48.983323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.030836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37860682088b49299e718f51c03daa8e', 'request_id': 'req-11ba215f-3c67-4d22-8fc8-891998959935'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:49.037555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37860682088b49299e718f51c03daa8e', 'request_id': 'req-11ba215f-3c67-4d22-8fc8-891998959935'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:49.044646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37860682088b49299e718f51c03daa8e', 'request_id': 'req-11ba215f-3c67-4d22-8fc8-891998959935'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:49.057485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] The token cache is being invalidate because project 37860682088b49299e718f51c03daa8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:49.057780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:49.058436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-11ba215f-3c67-4d22-8fc8-891998959935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:49.059961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 831/1662] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:48 2022] DELETE /identity/v3/projects/37860682088b49299e718f51c03daa8e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:42:49.064786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.065255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.065344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.087878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:49.089091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:49.151874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 832/1663] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:49.160899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.161427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.161569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.183805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:49.185046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:49.193171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 832/1664] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:49.254552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.264656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058a983-6cd3-41db-859f-d0d0a4cf73ea None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.269604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.270210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.270493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.270765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.271490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:49.271794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.273444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.286187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 833/1665] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:42:49.291097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.300609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd39971-65be-457a-ac9b-5019629718bd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.305887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.306503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.306783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.307056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.307735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.309658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.338920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 833/1666] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:49 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:49.343307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.354028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d1e14-d834-417e-ae9a-0ecf44bfe4ed admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.416295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.416973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.417246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.417516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.418641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.420178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.443875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 834/1667] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:49 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:49.448280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.458955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad97e-1f79-4d01-a194-d52535b0c99c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.463665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.464296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.464553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.464824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.465467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.467092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.478010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 834/1668] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:49.482200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.493259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f8a5a-ea2d-41ee-800b-c49f52909e59 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.498320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ee475-f2bf-4d6b-ad18-031d50818eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.499011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ee475-f2bf-4d6b-ad18-031d50818eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.499478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ee475-f2bf-4d6b-ad18-031d50818eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.499771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ee475-f2bf-4d6b-ad18-031d50818eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.500426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ee475-f2bf-4d6b-ad18-031d50818eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.502044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ee475-f2bf-4d6b-ad18-031d50818eb1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.513523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 835/1669] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:49.517936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.528511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ba2c1-e4de-4b65-a411-91ce9783d9c6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.533466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.534128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.534404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.534674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] PATH_INFO: `/v3/projects/bc43c2d6c1364f23b21d203467adb5c5/users/0a48c619ff834ca7942d9be275fe61ea/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.551868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc43c2d6c1364f23b21d203467adb5c5, user_id=0a48c619ff834ca7942d9be275fe61ea, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.554777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.570161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 835/1670] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:49 2022] PUT /identity/v3/projects/bc43c2d6c1364f23b21d203467adb5c5/users/0a48c619ff834ca7942d9be275fe61ea/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:49.575480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceed0b-412a-463f-9937-bf50115a6514 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.575736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceed0b-412a-463f-9937-bf50115a6514 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.576101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceed0b-412a-463f-9937-bf50115a6514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.605109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9cceed0b-412a-463f-9937-bf50115a6514 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:49.606330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9cceed0b-412a-463f-9937-bf50115a6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:49.667643 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 836/1671] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:42:49.678902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:49.680783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:49.758258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:49.792204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1a1b-60c3-4c25-885e-d2a494df850b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:49.797497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:49.800086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:49.800504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:49.800881 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:49.812147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:49.813671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:49.826573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 836/1672] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:50.692627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6b83-f9b8-4e33-a148-3bfb703eb7ca None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:50.693090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6b83-f9b8-4e33-a148-3bfb703eb7ca None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:50.693347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec6b83-f9b8-4e33-a148-3bfb703eb7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:50.702151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:50.725642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-36ec6b83-f9b8-4e33-a148-3bfb703eb7ca None None] MFA Rules not processed for user `0a48c619ff834ca7942d9be275fe61ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:50.727003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-36ec6b83-f9b8-4e33-a148-3bfb703eb7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:50.830359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 837/1673] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:50.859149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:50.865845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:50.978305 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f54295-90bc-4681-8f16-b05635b6e594 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:50.984690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:50.985320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:50.985601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:50.985856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:51.010704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:51.012910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:51.052548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 837/1674] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:52.578298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad1857-f090-4fc5-9764-ec7ee5c79e8d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:52.578831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad1857-f090-4fc5-9764-ec7ee5c79e8d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:52.578831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad1857-f090-4fc5-9764-ec7ee5c79e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:52.598121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d0ad1857-f090-4fc5-9764-ec7ee5c79e8d None None] MFA Rules not processed for user `0a48c619ff834ca7942d9be275fe61ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:52.599953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d0ad1857-f090-4fc5-9764-ec7ee5c79e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:52.606866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 838/1675] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:56.489390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:56.491462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:56.504857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18065017-1f8b-4276-ba09-13ac54ecce11 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:56.509956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:56.510573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:56.510850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:56.511119 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] PATH_INFO: `/v3/users/0cd91022635647bbbb3324818a5e04d4` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:56.512288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd91022635647bbbb3324818a5e04d4)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:56.514198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:56.566424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd91022635647bbbb3324818a5e04d4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:56.566424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd91022635647bbbb3324818a5e04d4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:56.569366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd91022635647bbbb3324818a5e04d4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:56.569989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd91022635647bbbb3324818a5e04d4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:56.577413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd91022635647bbbb3324818a5e04d4'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:56.594686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 838/1676] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:56 2022] DELETE /identity/v3/users/0cd91022635647bbbb3324818a5e04d4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:56.599116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:56.599630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:56.599741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:56.620765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:56.621959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:56.679118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 839/1677] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:56.688828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:56.690213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:56.748866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46659cf-5146-4f80-99fd-3c481f893d10 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:56.752448 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:56.753079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:56.753358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:56.753629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:56.762728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:56.764179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:56.771875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:56.780564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 839/1678] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:57.042547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:57.044009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:57.057637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b6167-111e-4be1-8bdd-b51c015fe45e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:57.062536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:57.063208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:57.063518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:57.063800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] PATH_INFO: `/v3/projects/2791cf70e78a453888438f37b9564896` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:57.065117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2791cf70e78a453888438f37b9564896)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:57.067015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:57.115258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2791cf70e78a453888438f37b9564896', 'request_id': 'req-1eb0dc0d-7148-462e-a8c7-13abec8c932d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.123735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2791cf70e78a453888438f37b9564896', 'request_id': 'req-1eb0dc0d-7148-462e-a8c7-13abec8c932d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.130176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2791cf70e78a453888438f37b9564896', 'request_id': 'req-1eb0dc0d-7148-462e-a8c7-13abec8c932d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.140474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] The token cache is being invalidate because project 2791cf70e78a453888438f37b9564896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:57.140757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.141376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.142877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 840/1679] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:57 2022] DELETE /identity/v3/projects/2791cf70e78a453888438f37b9564896 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:57.147465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:57.157143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baee451-0ed3-4c95-8cca-4671218002d3 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:57.213622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:57.214223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:57.214530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:57.214799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] PATH_INFO: `/v3/users/8afde52b50d6482eb37d9a9e4d91d098` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:57.215947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afde52b50d6482eb37d9a9e4d91d098)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:57.217879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:57.263395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afde52b50d6482eb37d9a9e4d91d098'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.263980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afde52b50d6482eb37d9a9e4d91d098'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.270715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afde52b50d6482eb37d9a9e4d91d098'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.271328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afde52b50d6482eb37d9a9e4d91d098'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.278926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-543786b5-cc23-4325-95da-4763beabc2d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afde52b50d6482eb37d9a9e4d91d098'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.296132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 840/1680] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:42:57 2022] DELETE /identity/v3/users/8afde52b50d6482eb37d9a9e4d91d098 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:57.664339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:57.666231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:57.683635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0dc0d-7148-462e-a8c7-13abec8c932d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:57.691782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:57.744085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:57.744716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:57.744997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:57.745264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] PATH_INFO: `/v3/projects/be1d333030914033afb2b69d31e6e14b` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:57.746371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1d333030914033afb2b69d31e6e14b)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:57.748314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:57.793907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1d333030914033afb2b69d31e6e14b', 'request_id': 'req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.802441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1d333030914033afb2b69d31e6e14b', 'request_id': 'req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.810447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1d333030914033afb2b69d31e6e14b', 'request_id': 'req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.822254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] The token cache is being invalidate because project be1d333030914033afb2b69d31e6e14b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:42:57.822541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.823232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f7b1391f-cf00-4191-bf7c-fa4c4e42ee8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:42:57.825069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 841/1681] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:42:57 2022] DELETE /identity/v3/projects/be1d333030914033afb2b69d31e6e14b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:57.829807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:57.830265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:57.830351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:57.849791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:57.851011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:57.910979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 841/1682] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:57.920055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:57.920425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:57.920766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:57.940577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:57.941771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:57.950146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 842/1683] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:57.954261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:57.964114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1649e0-4671-4052-95f9-268b1dc2ce22 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:57.968602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:57.969192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:57.969514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:57.969781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:57.970415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:42:57.970707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:57.972314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:57.983588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 842/1684] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:42:57.987735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:57.997413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5a796-fa03-42bb-919b-3c4d4a29b185 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.002076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.002666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.002944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.003242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.003900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.005782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.032724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 843/1685] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:57 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:58.036911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.043569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:58.047329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15452863-b5e0-49d6-b871-eeb340dc0f2b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.103932 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.104540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.104820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.105089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.105719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.107642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.130927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 843/1686] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:58 2022] POST /identity/v3/users => generated 278 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:58.135159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.145391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59a79-2854-4ce5-9048-175964cb77b7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.150029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.150633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.150910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.151180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.151856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.153435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.164262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 844/1687] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:58.168163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.178758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acfec9-a7ae-4b5a-99fb-20fd2e7380e1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.183361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb23c9f-a47a-433c-ac52-683c1881ef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.183983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb23c9f-a47a-433c-ac52-683c1881ef2d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.184262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb23c9f-a47a-433c-ac52-683c1881ef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.184533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb23c9f-a47a-433c-ac52-683c1881ef2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.185176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb23c9f-a47a-433c-ac52-683c1881ef2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.186792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb23c9f-a47a-433c-ac52-683c1881ef2d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.199239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 844/1688] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:58.203358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.213188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79812af9-b1e9-41f6-9927-d997a2c7b247 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.217735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.218374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.218649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.218921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] PATH_INFO: `/v3/projects/79bca8c0ede24491ba5201746bb9adc3/users/5c54b93feb334d099257f3b676bfda17/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.233598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bca8c0ede24491ba5201746bb9adc3, user_id=5c54b93feb334d099257f3b676bfda17, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.236273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.249489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 845/1689] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:58 2022] PUT /identity/v3/projects/79bca8c0ede24491ba5201746bb9adc3/users/5c54b93feb334d099257f3b676bfda17/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:58.259784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.260073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.260350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.288806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] MFA Rules not processed for user `5c54b93feb334d099257f3b676bfda17`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:58.289926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:58.352825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 845/1690] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:58.357212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.396748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:58.424757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cfb35-84f6-42fb-872a-85821ea2332c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.429905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.430509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.430790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.431059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.431789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.433686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.459837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 846/1691] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:42:58 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:42:58.463927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.474664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa11eac-1129-4bb6-8f3c-e1edc55b1aea None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.530345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.530942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.531215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.531511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.532140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.534045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.555989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 846/1692] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:42:58 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:42:58.560105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.570869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b65b4c-2cc3-4e61-a1c9-62aa69b25193 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.575787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.576377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.576651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.576916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.577550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.579117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.589398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 847/1693] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:58.593563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.598689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:58.605133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5a66a-f514-465d-8da0-569c76e8a481 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.610179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.610943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.611225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.611531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] PATH_INFO: `/v3/projects/4a859438ec014a83981db2e8625b6322/users/8df5f0e3affd4130a5d51e8be2eba92d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.626854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a859438ec014a83981db2e8625b6322, user_id=8df5f0e3affd4130a5d51e8be2eba92d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.629539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.642914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 847/1694] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:58 2022] PUT /identity/v3/projects/4a859438ec014a83981db2e8625b6322/users/8df5f0e3affd4130a5d51e8be2eba92d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:58.646950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.657033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8782e-aa14-48ad-be04-8e0a7e95e333 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.715560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60da9d6-5ba3-4972-856b-053639284939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.716262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60da9d6-5ba3-4972-856b-053639284939 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.716455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60da9d6-5ba3-4972-856b-053639284939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.716728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60da9d6-5ba3-4972-856b-053639284939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.717379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60da9d6-5ba3-4972-856b-053639284939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.718975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60da9d6-5ba3-4972-856b-053639284939 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.729417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 848/1695] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:42:58.733725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.745967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5873a9f-a258-4b45-b108-7c47d2f95b1a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.751149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.751845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.752128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.752404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] PATH_INFO: `/v3/projects/4a859438ec014a83981db2e8625b6322/users/8df5f0e3affd4130a5d51e8be2eba92d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.754693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a859438ec014a83981db2e8625b6322, user_id=8df5f0e3affd4130a5d51e8be2eba92d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.758210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.769278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 848/1696] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:42:58 2022] PUT /identity/v3/projects/4a859438ec014a83981db2e8625b6322/users/8df5f0e3affd4130a5d51e8be2eba92d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:42:58.780730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.781018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.781295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.821230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] MFA Rules not processed for user `8df5f0e3affd4130a5d51e8be2eba92d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:42:58.822455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:58.895599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 849/1697] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:42:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:42:58.909825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:58.911617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:42:58.968503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e71b6c-6f6f-4a82-ac84-aa8930ace442 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:58.972082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:58.972676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:58.972957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:58.973229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:58.982614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:58.984062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:58.993764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 849/1698] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:42:59.067001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:42:59.076551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464a229-9d05-4d10-ba53-26a3b396cd0f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:42:59.080179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ee36-5061-47a3-888b-1e07a3399bec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:42:59.080769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ee36-5061-47a3-888b-1e07a3399bec service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:42:59.081042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ee36-5061-47a3-888b-1e07a3399bec service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:42:59.081309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ee36-5061-47a3-888b-1e07a3399bec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:42:59.110098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:42:59.138410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466ee36-5061-47a3-888b-1e07a3399bec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:42:59.140082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466ee36-5061-47a3-888b-1e07a3399bec service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:42:59.150132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 850/1699] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:42:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:08.989110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=40,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:09.145294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=13 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:11.089387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:11.090811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:11.157517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59458-326b-498a-9dd1-7d583eb09bc0 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:11.162005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:11.162626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.162903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.163170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] PATH_INFO: `/v3/users/d464c945bf954016b8f7291b659cabbe` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.164315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d464c945bf954016b8f7291b659cabbe)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:11.166226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:11.211794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd464c945bf954016b8f7291b659cabbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.212444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd464c945bf954016b8f7291b659cabbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.219395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd464c945bf954016b8f7291b659cabbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.220053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd464c945bf954016b8f7291b659cabbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.227107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd464c945bf954016b8f7291b659cabbe'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.244843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 850/1700] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:11 2022] DELETE /identity/v3/users/d464c945bf954016b8f7291b659cabbe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:11.249257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.249715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.249828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.269162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:11.270371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:11.327198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 851/1701] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:11.337126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:11.338514 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:11.395664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86e5f0-e95e-4b00-9cdc-84ae9a60b141 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:11.399737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba170f-38a5-4578-90e1-4cd8e76c5c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:11.400338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba170f-38a5-4578-90e1-4cd8e76c5c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.400615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba170f-38a5-4578-90e1-4cd8e76c5c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.400890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba170f-38a5-4578-90e1-4cd8e76c5c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.410327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba170f-38a5-4578-90e1-4cd8e76c5c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:11.411799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba170f-38a5-4578-90e1-4cd8e76c5c38 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:11.422285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 851/1702] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:11.688472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:11.690006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:11.702612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52bf25-c22c-40fa-9298-684c82cba01d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:11.707677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:11.708369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.708648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.708918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] PATH_INFO: `/v3/projects/991e976345ea4a988bbe706fdd27766e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.711747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=991e976345ea4a988bbe706fdd27766e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:11.713691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:11.760770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991e976345ea4a988bbe706fdd27766e', 'request_id': 'req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.768082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991e976345ea4a988bbe706fdd27766e', 'request_id': 'req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.775300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991e976345ea4a988bbe706fdd27766e', 'request_id': 'req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.786195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] The token cache is being invalidate because project 991e976345ea4a988bbe706fdd27766e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:11.786510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.787173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d4c0b5ca-b68f-494e-acfa-55bf69df66dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:11.788677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 852/1703] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:11 2022] DELETE /identity/v3/projects/991e976345ea4a988bbe706fdd27766e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:11.795340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.795666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.795962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.815248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:11.816486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:11.845092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:11.900519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 852/1704] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:11.908754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.909056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.909334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.931812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:11.933088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:11.946487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 853/1705] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:11.947343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:11.957552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7459e58-49b4-484b-8587-ec1b23bf4cbd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:11.963449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:11.964064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:11.964376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:11.964650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:11.965306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:11.965606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:11.967218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:11.983214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 853/1706] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:43:11.987613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:12.001930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05778d-37c4-414a-a7ec-30b7da1a5e68 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:12.011957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:12.012581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.012863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.013136 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.013780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:12.015715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:12.047665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 854/1707] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:11 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:12.051901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:12.063065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c909f-8ad5-46c1-a8bf-1ae955486786 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:12.121692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:12.122339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.122598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.122856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.123532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:12.125456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:12.147021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 854/1708] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:12 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:12.151184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:12.162052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac0b9-c556-4718-8106-5e8e0cf65e40 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:12.167193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:12.167836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.168118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.168391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.169042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:12.170628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:12.182420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 855/1709] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:12.186433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:12.197879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d619b22-f9d3-4368-b886-b7964629cde5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:12.205086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af20789-0bdf-4b56-9193-6efbc2581589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:12.206363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af20789-0bdf-4b56-9193-6efbc2581589 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.207054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af20789-0bdf-4b56-9193-6efbc2581589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.207832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af20789-0bdf-4b56-9193-6efbc2581589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.209143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af20789-0bdf-4b56-9193-6efbc2581589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:12.211896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af20789-0bdf-4b56-9193-6efbc2581589 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:12.224733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 855/1710] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:12.229218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:12.239750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bbc51-a800-488c-b54c-ca079ba1095b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:12.245180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:12.245879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.246279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.246561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] PATH_INFO: `/v3/projects/d6542df65cd04595815520380e97ba3d/users/f0669c12662542059a186133fb1e2c91/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.269303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6542df65cd04595815520380e97ba3d, user_id=f0669c12662542059a186133fb1e2c91, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:12.272046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:12.288671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 856/1711] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:12 2022] PUT /identity/v3/projects/d6542df65cd04595815520380e97ba3d/users/f0669c12662542059a186133fb1e2c91/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:12.298268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.298608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.298902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.332803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] MFA Rules not processed for user `f0669c12662542059a186133fb1e2c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:12.334306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:12.415214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 856/1712] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:12.426763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:12.429671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:12.457595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:12.494453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5740d-8b46-4760-a674-a28d7b171ae2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:12.498044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f9943-a8b6-4339-a388-34ccf605b89c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:12.498652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9943-a8b6-4339-a388-34ccf605b89c service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:12.498929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9943-a8b6-4339-a388-34ccf605b89c service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:12.499198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9943-a8b6-4339-a388-34ccf605b89c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:12.507301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f9943-a8b6-4339-a388-34ccf605b89c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:12.508743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f9943-a8b6-4339-a388-34ccf605b89c service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:12.517354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 857/1713] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:22.410957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:22.513392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=17 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:22.578704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:22.580149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:22.646574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddf526-6a19-4203-a814-940b5fdbef37 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:22.651031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:22.652258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:22.652529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:22.652799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] PATH_INFO: `/v3/users/5c54b93feb334d099257f3b676bfda17` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:22.653917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c54b93feb334d099257f3b676bfda17)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:22.656131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:22.702580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c54b93feb334d099257f3b676bfda17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:22.703233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c54b93feb334d099257f3b676bfda17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:22.710225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c54b93feb334d099257f3b676bfda17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:22.710868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c54b93feb334d099257f3b676bfda17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:22.721017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c54b93feb334d099257f3b676bfda17'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:22.743488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 857/1714] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:22 2022] DELETE /identity/v3/users/5c54b93feb334d099257f3b676bfda17 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:22.748144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:22.748460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:22.748744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:22.770932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:22.773100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:22.870516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 858/1715] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:22.879970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:22.881384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:22.936886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ea2f-2f4c-4540-8522-67fcab22a785 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:22.940444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:22.941032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:22.941306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:22.941572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:22.950734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:22.952169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:22.961228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 858/1716] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:23.439957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:23.441411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:23.453699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6f665-9d6f-4fca-bbfb-3f5c96239fbb None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:23.458353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:23.458990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:23.459269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:23.459571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] PATH_INFO: `/v3/projects/79bca8c0ede24491ba5201746bb9adc3` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:23.460703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bca8c0ede24491ba5201746bb9adc3)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:23.462625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:23.508945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bca8c0ede24491ba5201746bb9adc3', 'request_id': 'req-a2dc59d0-92f5-405e-ab95-db77377b83d1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.517634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bca8c0ede24491ba5201746bb9adc3', 'request_id': 'req-a2dc59d0-92f5-405e-ab95-db77377b83d1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.524118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bca8c0ede24491ba5201746bb9adc3', 'request_id': 'req-a2dc59d0-92f5-405e-ab95-db77377b83d1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.534859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] The token cache is being invalidate because project 79bca8c0ede24491ba5201746bb9adc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:23.535148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.535854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.537379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 859/1717] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:23 2022] DELETE /identity/v3/projects/79bca8c0ede24491ba5201746bb9adc3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:23.541381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:23.552906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66427b9c-a8bf-496d-9e31-f38d808405a8 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:23.571737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:23.617287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:23.617922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:23.618206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:23.618461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] PATH_INFO: `/v3/users/8df5f0e3affd4130a5d51e8be2eba92d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:23.619611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df5f0e3affd4130a5d51e8be2eba92d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:23.622373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:23.671882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df5f0e3affd4130a5d51e8be2eba92d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.672609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df5f0e3affd4130a5d51e8be2eba92d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.680231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df5f0e3affd4130a5d51e8be2eba92d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.680869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df5f0e3affd4130a5d51e8be2eba92d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.688944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6b6590ff-0ae7-4831-850e-e64f43e0ad00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df5f0e3affd4130a5d51e8be2eba92d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:23.706836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 859/1718] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:23 2022] DELETE /identity/v3/users/8df5f0e3affd4130a5d51e8be2eba92d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:23.980708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:23.982281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:23.998621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc59d0-92f5-405e-ab95-db77377b83d1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.061845 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.062485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.062765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.063035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] PATH_INFO: `/v3/projects/4a859438ec014a83981db2e8625b6322` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.064188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a859438ec014a83981db2e8625b6322)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.066086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.096149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:24.113417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a859438ec014a83981db2e8625b6322', 'request_id': 'req-17fdfaec-2606-4ff4-9503-49539d926f44'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:24.120563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a859438ec014a83981db2e8625b6322', 'request_id': 'req-17fdfaec-2606-4ff4-9503-49539d926f44'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:24.126786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a859438ec014a83981db2e8625b6322', 'request_id': 'req-17fdfaec-2606-4ff4-9503-49539d926f44'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:24.137717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] The token cache is being invalidate because project 4a859438ec014a83981db2e8625b6322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:24.138007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:24.138634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-17fdfaec-2606-4ff4-9503-49539d926f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:24.140140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 860/1719] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:23 2022] DELETE /identity/v3/projects/4a859438ec014a83981db2e8625b6322 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:24.144848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.145133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.145406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.165441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:24.166673 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:24.224683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 860/1720] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:24.232951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.233232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.233501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.253289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:24.254494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:24.262849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 861/1721] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:24.266993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.277269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf469f4-a05b-4a6a-9658-91fcec49b3c8 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.282056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.282644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.282919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.283186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.283852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:24.284163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.285733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.297067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 861/1722] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:43:24.301125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.310027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b15dfe-9ff8-483f-aa34-b40bd11636da None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.314552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.315135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.315406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.315699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.316324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.318184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.344857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 862/1723] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:24.349012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.358761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90988c7-5ee6-4444-bf5a-640441dd738b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.383894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:24.418345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.418948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.419225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.419549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.420158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.422064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.442536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 862/1724] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:24.446747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.457425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7b531-6487-46df-a400-1117838f6df0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.462180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.462824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.463109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.463387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.464106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.465759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.475627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 863/1725] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:24.479905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.489594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea21b35-2cbc-4921-9558-9c282be2e59f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.494128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfa00a-bd3e-4201-88d6-775c64dc1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.494714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfa00a-bd3e-4201-88d6-775c64dc1b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.495568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfa00a-bd3e-4201-88d6-775c64dc1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.495839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfa00a-bd3e-4201-88d6-775c64dc1b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.496487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfa00a-bd3e-4201-88d6-775c64dc1b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.498045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfa00a-bd3e-4201-88d6-775c64dc1b5a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.509444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 863/1726] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:24.513454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.523273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241ffd3-f6d2-405c-801f-c7bbe63bbf6a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.528510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.529165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.529696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.529974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] PATH_INFO: `/v3/projects/99b43990bb5d4fbbb771e3628e2b300d/users/06213dae7ebb4435822d8853bad72e53/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.544783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b43990bb5d4fbbb771e3628e2b300d, user_id=06213dae7ebb4435822d8853bad72e53, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.547470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.559935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 864/1727] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:24 2022] PUT /identity/v3/projects/99b43990bb5d4fbbb771e3628e2b300d/users/06213dae7ebb4435822d8853bad72e53/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:24.570159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.570622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.570719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.597698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] MFA Rules not processed for user `06213dae7ebb4435822d8853bad72e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:24.598886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:24.663646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 864/1728] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:24.668999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.737067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f86dff-a851-4891-961b-5ba25ded9894 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.742702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.742816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.743198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.743365 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.744026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.745934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.773508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 865/1729] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:24.778090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.788938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae8847-b18b-4943-b13b-442d47982417 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.844865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.845592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.845876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.846144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.846776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.848740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.871452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 865/1730] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:24 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:24.875882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.889038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2292-1fee-4f4b-9d12-b84457d55352 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.894115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.894741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.895011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.895282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.895958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.897540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.915103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 866/1731] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:24.915292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.925633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837948-268c-4d2a-9ef1-4db9b1c02847 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:24.930406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:24.931070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:24.931351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:24.931652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] PATH_INFO: `/v3/projects/9333b4251ecc4f7c8805ba4241787b8c/users/9932b1629d6e4210b99f94262f0e0712/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:24.950009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9333b4251ecc4f7c8805ba4241787b8c, user_id=9932b1629d6e4210b99f94262f0e0712, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:24.952766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:24.968698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 866/1732] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:24 2022] PUT /identity/v3/projects/9333b4251ecc4f7c8805ba4241787b8c/users/9932b1629d6e4210b99f94262f0e0712/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:24.972390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:24.989313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a79f-3d4a-44f5-8901-2da409881b8e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:25.014154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:25.059554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37193dbe-725d-4df6-82ca-7343f63eb110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:25.060206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37193dbe-725d-4df6-82ca-7343f63eb110 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:25.060502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37193dbe-725d-4df6-82ca-7343f63eb110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:25.060769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37193dbe-725d-4df6-82ca-7343f63eb110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:25.064152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37193dbe-725d-4df6-82ca-7343f63eb110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:25.067041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37193dbe-725d-4df6-82ca-7343f63eb110 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:25.078259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 867/1733] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:25.082230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:25.088006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:25.093835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97eadd-d03c-4f9b-8a19-9ed728424dd4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:25.108581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:25.109329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:25.109686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:25.110034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] PATH_INFO: `/v3/projects/9333b4251ecc4f7c8805ba4241787b8c/users/9932b1629d6e4210b99f94262f0e0712/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:25.112180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9333b4251ecc4f7c8805ba4241787b8c, user_id=9932b1629d6e4210b99f94262f0e0712, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:25.115217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:25.126349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 867/1734] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:25 2022] PUT /identity/v3/projects/9333b4251ecc4f7c8805ba4241787b8c/users/9932b1629d6e4210b99f94262f0e0712/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:25.138642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:25.138931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:25.139212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:25.170832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] MFA Rules not processed for user `9932b1629d6e4210b99f94262f0e0712`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:25.172093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:25.250976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 868/1735] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:25.262533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:25.264277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:25.324187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49689eb4-a759-4ad9-92cc-0340727a5e04 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:25.328216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeee30-e066-4669-938e-fc123efa7827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:25.329127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeee30-e066-4669-938e-fc123efa7827 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:25.329201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeee30-e066-4669-938e-fc123efa7827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:25.329439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeee30-e066-4669-938e-fc123efa7827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:25.341635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeee30-e066-4669-938e-fc123efa7827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:25.343213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeee30-e066-4669-938e-fc123efa7827 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:25.368109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 868/1736] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:25.474508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:25.490797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483f000-8276-4aaa-9c8d-d2635667a9b4 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:25.495398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ecb26-008c-4e0b-8f78-77749671ed0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:25.496177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ecb26-008c-4e0b-8f78-77749671ed0b service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:25.496446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ecb26-008c-4e0b-8f78-77749671ed0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:25.496730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ecb26-008c-4e0b-8f78-77749671ed0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:25.576587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ecb26-008c-4e0b-8f78-77749671ed0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:25.579053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ecb26-008c-4e0b-8f78-77749671ed0b service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:25.590377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 869/1737] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:27.410161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c601156-1ee1-4089-baeb-5668a5a5aee9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:27.410948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c601156-1ee1-4089-baeb-5668a5a5aee9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:27.411716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c601156-1ee1-4089-baeb-5668a5a5aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:27.431490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2c601156-1ee1-4089-baeb-5668a5a5aee9 None None] MFA Rules not processed for user `0a48c619ff834ca7942d9be275fe61ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:27.433387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2c601156-1ee1-4089-baeb-5668a5a5aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:27.489090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 869/1738] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:30.955638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19747-9ed9-4c31-9a00-43aa9aacc7cf None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:30.956154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19747-9ed9-4c31-9a00-43aa9aacc7cf None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:30.956437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19747-9ed9-4c31-9a00-43aa9aacc7cf None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:30.958285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 870/1739] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:43:30.967453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f1d51-f60a-40fa-89f5-11d9ec735958 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:30.967936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f1d51-f60a-40fa-89f5-11d9ec735958 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:30.968124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f1d51-f60a-40fa-89f5-11d9ec735958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:30.988979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4b4f1d51-f60a-40fa-89f5-11d9ec735958 None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:30.990324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b4f1d51-f60a-40fa-89f5-11d9ec735958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:31.062214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 870/1740] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:43:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:31.255216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07a49e-af3b-43be-a463-27e2b81ee5ab None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.255576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07a49e-af3b-43be-a463-27e2b81ee5ab None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.256077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07a49e-af3b-43be-a463-27e2b81ee5ab None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.257817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 871/1741] 10.222.0.39 () {54 vars in 961 bytes} [Tue Oct 11 06:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:43:31.265580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.265866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.266135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.278042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07a49e-af3b-43be-a463-27e2b81ee5ab None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:31.280461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5d07a49e-af3b-43be-a463-27e2b81ee5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:31.292537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:31.293824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:31.303035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 871/1742] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Oct 11 06:43:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:31.312450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:31.322191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771d79f-1063-499c-bf43-e6adc2965bdd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:31.327270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:31.327900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.328179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.328454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.331162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:31.337241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:31.339352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:31.348853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 872/1743] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:31.351292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07a49e-af3b-43be-a463-27e2b81ee5ab None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:31.356400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:31.357033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.357313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.357589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] PATH_INFO: `/v3/users/e97edc31dd834fd8811ad1bb25f1168b` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.358726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97edc31dd834fd8811ad1bb25f1168b)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:31.360682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:31.411938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97edc31dd834fd8811ad1bb25f1168b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.412650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97edc31dd834fd8811ad1bb25f1168b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.413297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97edc31dd834fd8811ad1bb25f1168b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.421047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97edc31dd834fd8811ad1bb25f1168b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.429181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-04621963-d3d5-44f7-bca8-277af06089fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97edc31dd834fd8811ad1bb25f1168b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.446926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 872/1744] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:31 2022] DELETE /identity/v3/users/e97edc31dd834fd8811ad1bb25f1168b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:43:31.625197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:31.636966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651351b-7fb6-4e7b-8df4-bb3e784cb2b2 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:31.651409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:31.701812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:31.702353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.702631 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.702903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] PATH_INFO: `/v3/projects/048f1b5e89c8417695e7916b444e2778` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.704022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048f1b5e89c8417695e7916b444e2778)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:31.705933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:31.751582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048f1b5e89c8417695e7916b444e2778', 'request_id': 'req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.757655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048f1b5e89c8417695e7916b444e2778', 'request_id': 'req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.765129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048f1b5e89c8417695e7916b444e2778', 'request_id': 'req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.775913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] The token cache is being invalidate because project 048f1b5e89c8417695e7916b444e2778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:31.776196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.776824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c1f8069-5ac0-4e1f-9a15-0ee0112820a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:31.778293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 873/1745] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:31 2022] DELETE /identity/v3/projects/048f1b5e89c8417695e7916b444e2778 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:31.784517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.784825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.785120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.804665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:31.805851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:31.863406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 873/1746] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:31.871642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.871928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.872201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.893069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:31.894278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:31.903073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 874/1747] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:31.907519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:31.917444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3af74-0952-42c9-a969-d7ddd4697dd4 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:31.921952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:31.922546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.922823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.923090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.923771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:31.924071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:31.925673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:31.941921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 874/1748] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:43:31.942931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:31.954004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6d6bf-0b2d-4833-906b-a3fa8a161bb5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:31.958963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:31.959728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:31.960166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:31.960479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:31.961471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:31.963400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:31.991668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 875/1749] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:31 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:31.996933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.007295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6fdab-b699-45c8-b1d0-8b1fcaf300dc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.060177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:32.076165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.076795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.077103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.077379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.078023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.080058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.102212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 875/1750] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:31 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:32.106757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.117875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8f38d-d52f-4631-b475-41ad2fdfef9d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.122996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.123803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.124079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.124480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.125186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.126854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.138778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 876/1751] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:32.142637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.157123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55761c12-bd64-472f-9c81-03686925011c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.163261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c3f93-34f9-4dcb-b1bb-f4b1e20b3f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.164262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c3f93-34f9-4dcb-b1bb-f4b1e20b3f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.164575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c3f93-34f9-4dcb-b1bb-f4b1e20b3f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.164844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c3f93-34f9-4dcb-b1bb-f4b1e20b3f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.165557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c3f93-34f9-4dcb-b1bb-f4b1e20b3f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.168335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c3f93-34f9-4dcb-b1bb-f4b1e20b3f25 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.181420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 876/1752] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:43:32.185670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.198333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6c49-eae5-405c-881f-efe9bd4ca299 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.206865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.207901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.208197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.208473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] PATH_INFO: `/v3/projects/f5b75b666c884d09851cca0c0730a1f0/users/e4ebebb143424d6bb500909951ce87ce/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.230458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b75b666c884d09851cca0c0730a1f0, user_id=e4ebebb143424d6bb500909951ce87ce, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.233332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.254801 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 877/1753] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:32 2022] PUT /identity/v3/projects/f5b75b666c884d09851cca0c0730a1f0/users/e4ebebb143424d6bb500909951ce87ce/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:32.267871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.269252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.269582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.323756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] MFA Rules not processed for user `e4ebebb143424d6bb500909951ce87ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:32.326555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:32.341404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.343463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:32.409523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:32.427832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 877/1754] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:32.432148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.444522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750837e7-96a4-450f-b7c6-c5ef80608258 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.446989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb131272-0b5b-4625-90e4-b657ae6a832f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.449975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.450661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.451002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.451332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.452072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.454270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.454388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.455607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.455832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.456112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] PATH_INFO: `/v3/users/f0669c12662542059a186133fb1e2c91` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.457301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0669c12662542059a186133fb1e2c91)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.460088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.502018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 878/1755] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:32 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:32.503677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.558007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0669c12662542059a186133fb1e2c91'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:32.558802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0669c12662542059a186133fb1e2c91'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:32.570130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0669c12662542059a186133fb1e2c91'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:32.574942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0669c12662542059a186133fb1e2c91'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:32.585187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d4ade306-6d61-4b4f-9899-5135d4dfac75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0669c12662542059a186133fb1e2c91'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:32.616833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55b98-c26a-4cb9-bbfd-bda299071721 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.625182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.626296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.627392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.627920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.628601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.630620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.638250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 878/1756] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:32 2022] DELETE /identity/v3/users/f0669c12662542059a186133fb1e2c91 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:32.643957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.644505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.646037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.661678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 879/1757] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:32 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:32.665399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.667620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:32.681894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8246de-3e0e-4807-badf-aafb2563da62 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.684556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:32.686674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:32.710401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:32.771328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.772144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.772491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.772843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.773575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.775548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.789914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 880/1758] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:43:32.792960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 879/1759] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:32.795182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.803963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.805510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:32.814388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac227-21ac-4075-a33c-9e74537271ca admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.821474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.822231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.822591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.822965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.823721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.826015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.840972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 881/1760] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:32.845180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:32.859172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435cfa0-97c2-413c-8fbe-6131a669ec72 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.864613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0568eb-acaf-4f12-9956-390d57216b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.865676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0568eb-acaf-4f12-9956-390d57216b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.865935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0568eb-acaf-4f12-9956-390d57216b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.866229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0568eb-acaf-4f12-9956-390d57216b0c admin admin] PATH_INFO: `/v3/projects/dd67fa0b8c34463c8e0ca7d9ae3cb1e0/users/73473a9d52214d34b58606c1fcfacb61/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.883051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a3c9-68a3-44e0-99f6-8dfbf80fc0f3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:32.883264 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0568eb-acaf-4f12-9956-390d57216b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd67fa0b8c34463c8e0ca7d9ae3cb1e0, user_id=73473a9d52214d34b58606c1fcfacb61, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.886077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0568eb-acaf-4f12-9956-390d57216b0c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.887052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:32.887699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.887979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.888506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.899797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:32.902016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:32.902565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 882/1761] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:32 2022] PUT /identity/v3/projects/dd67fa0b8c34463c8e0ca7d9ae3cb1e0/users/73473a9d52214d34b58606c1fcfacb61/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:43:32.912829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:32.913195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:32.913548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:32.923594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 880/1762] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:43:32.946017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] MFA Rules not processed for user `73473a9d52214d34b58606c1fcfacb61`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:32.947840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:33.033384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 883/1763] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:33.038458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.044491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:33.053138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd868fd-0457-4987-9495-128e86823e6f service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.127252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.127922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.128197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.128505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.129109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.131033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.160622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 881/1764] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:33 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:33.164725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.175813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4818b-75af-4b8d-9d59-32a5bd702a91 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.238957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.239527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.239811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.240085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.240734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.242663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.270619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 884/1765] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:33 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:33.271888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.282725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206b530-bb6e-4647-92ff-cb091169324d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.287588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.288272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.288624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.289567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.289683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.291356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.303155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 882/1766] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:33.308002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.318040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9498b-b199-4d50-91cb-692b25aba261 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.322617 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.323300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.323612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.323887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] PATH_INFO: `/v3/projects/9295b06bb80741ac849cdb2ed0c2f30f/users/6472fc3e67374873a44f193e9a6e4983/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.326142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:33.340667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9295b06bb80741ac849cdb2ed0c2f30f, user_id=6472fc3e67374873a44f193e9a6e4983, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.343317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.357307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 885/1767] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:33 2022] PUT /identity/v3/projects/9295b06bb80741ac849cdb2ed0c2f30f/users/6472fc3e67374873a44f193e9a6e4983/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:33.361798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.373638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fcf60-a5cc-44e5-acb5-25ba3524a251 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.406205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.410303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:33.454431 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.455035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.455314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.455687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.456253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.457852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.471959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 883/1768] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:33.472763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.483704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a78e-c7f2-4022-84a2-db5dce21944a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.490406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a7ca0-c1ce-488c-8268-82fa041f246d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.491136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a7ca0-c1ce-488c-8268-82fa041f246d admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.491278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312c378-48d0-4fed-981b-deabcfc8c3eb admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.491531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a7ca0-c1ce-488c-8268-82fa041f246d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.491726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a7ca0-c1ce-488c-8268-82fa041f246d admin admin] PATH_INFO: `/v3/projects/9295b06bb80741ac849cdb2ed0c2f30f/users/6472fc3e67374873a44f193e9a6e4983/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.493670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a7ca0-c1ce-488c-8268-82fa041f246d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9295b06bb80741ac849cdb2ed0c2f30f, user_id=6472fc3e67374873a44f193e9a6e4983, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.497001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a7ca0-c1ce-488c-8268-82fa041f246d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.497116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.498015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.498391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.498752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] PATH_INFO: `/v3/projects/d6542df65cd04595815520380e97ba3d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.500017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6542df65cd04595815520380e97ba3d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.501943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.508254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 884/1769] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:33 2022] PUT /identity/v3/projects/9295b06bb80741ac849cdb2ed0c2f30f/users/6472fc3e67374873a44f193e9a6e4983/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:33.519016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.519327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.520096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.556420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] MFA Rules not processed for user `6472fc3e67374873a44f193e9a6e4983`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:33.558358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:33.576851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6542df65cd04595815520380e97ba3d', 'request_id': 'req-62401d2c-7a4c-4aac-ac50-0689407d626f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:33.586670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6542df65cd04595815520380e97ba3d', 'request_id': 'req-62401d2c-7a4c-4aac-ac50-0689407d626f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:33.609644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6542df65cd04595815520380e97ba3d', 'request_id': 'req-62401d2c-7a4c-4aac-ac50-0689407d626f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:33.625759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:33.631191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] The token cache is being invalidate because project d6542df65cd04595815520380e97ba3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:33.631668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:33.632510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:33.634796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 886/1770] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:33 2022] DELETE /identity/v3/projects/d6542df65cd04595815520380e97ba3d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:33.676971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 885/1771] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:33.692497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.693921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:33.839683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62401d2c-7a4c-4aac-ac50-0689407d626f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.845358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.846438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.846920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.848587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.861989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:33.863539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:33.877067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 887/1772] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:33.925983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:33.936640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac334eaf-fed1-4d30-a1e3-70a9515f113f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:33.941350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:33.941957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:33.942229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:33.942494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:33.944312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:34.020495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:34.020810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0315cd5-2b63-49bc-8106-33d4352818c8 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:34.033257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 886/1773] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:35.358831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:35.360218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:35.378219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:35.426811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bc84f-a25e-4658-a39e-046605569255 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:35.431340 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:35.432017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:35.432294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:35.432567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] PATH_INFO: `/v3/users/06213dae7ebb4435822d8853bad72e53` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:35.433675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=06213dae7ebb4435822d8853bad72e53)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:35.435622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:35.482665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06213dae7ebb4435822d8853bad72e53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:35.483312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06213dae7ebb4435822d8853bad72e53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:35.483943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06213dae7ebb4435822d8853bad72e53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:35.490825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06213dae7ebb4435822d8853bad72e53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:35.498971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06213dae7ebb4435822d8853bad72e53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:35.517793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 888/1774] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:35 2022] DELETE /identity/v3/users/06213dae7ebb4435822d8853bad72e53 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:35.522234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:35.522734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:35.522845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:35.544022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:35.545225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:35.610375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 887/1775] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:35.620609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:35.622316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:35.729658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bda14-bcd3-426c-9086-3b4ae3e25bef admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:35.733731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:35.733986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:35.734226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:35.734495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:35.744177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:35.746196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:35.756072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 889/1776] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:36.068761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:36.070288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:36.082984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45fa06-c659-4a66-bb4e-f386b30f3491 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:36.087825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:36.089121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.089505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.089869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] PATH_INFO: `/v3/projects/99b43990bb5d4fbbb771e3628e2b300d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.092070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b43990bb5d4fbbb771e3628e2b300d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:36.093850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:36.147979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b43990bb5d4fbbb771e3628e2b300d', 'request_id': 'req-f65dbe4b-0a80-4278-a84d-75d17b041412'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.156457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b43990bb5d4fbbb771e3628e2b300d', 'request_id': 'req-f65dbe4b-0a80-4278-a84d-75d17b041412'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.164968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b43990bb5d4fbbb771e3628e2b300d', 'request_id': 'req-f65dbe4b-0a80-4278-a84d-75d17b041412'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.178549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] The token cache is being invalidate because project 99b43990bb5d4fbbb771e3628e2b300d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:36.178885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.179612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.181332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 888/1777] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:36 2022] DELETE /identity/v3/projects/99b43990bb5d4fbbb771e3628e2b300d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:36.185364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:36.196176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff5054-d251-4c4e-a132-127798a6365f service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:36.231224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:36.257403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:36.258089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.258375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.258649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] PATH_INFO: `/v3/users/9932b1629d6e4210b99f94262f0e0712` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.259819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9932b1629d6e4210b99f94262f0e0712)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:36.262281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:36.314940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9932b1629d6e4210b99f94262f0e0712'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.315578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9932b1629d6e4210b99f94262f0e0712'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.316202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9932b1629d6e4210b99f94262f0e0712'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.325993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9932b1629d6e4210b99f94262f0e0712'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.333654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f32f94c0-307a-4201-8104-b01848fa7296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9932b1629d6e4210b99f94262f0e0712'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.351889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 890/1778] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:36 2022] DELETE /identity/v3/users/9932b1629d6e4210b99f94262f0e0712 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:36.614943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:36.616426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:36.630493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dbe4b-0a80-4278-a84d-75d17b041412 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:36.634427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:36.691035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:36.691699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.691978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.692244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] PATH_INFO: `/v3/projects/9333b4251ecc4f7c8805ba4241787b8c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.693343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9333b4251ecc4f7c8805ba4241787b8c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:36.695239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:36.741978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9333b4251ecc4f7c8805ba4241787b8c', 'request_id': 'req-65b262e2-44fc-407f-941a-23caf40af1c2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.748947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9333b4251ecc4f7c8805ba4241787b8c', 'request_id': 'req-65b262e2-44fc-407f-941a-23caf40af1c2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.756812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9333b4251ecc4f7c8805ba4241787b8c', 'request_id': 'req-65b262e2-44fc-407f-941a-23caf40af1c2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.767876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] The token cache is being invalidate because project 9333b4251ecc4f7c8805ba4241787b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:36.768159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.768773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-65b262e2-44fc-407f-941a-23caf40af1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:36.770236 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 889/1779] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:36 2022] DELETE /identity/v3/projects/9333b4251ecc4f7c8805ba4241787b8c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:43:36.775059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.775550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.775711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.799541 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:36.800760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:36.865521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 891/1780] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:36.873999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.874294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.874561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.895259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:36.896516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:36.904850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 890/1781] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:36.909081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:36.919478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d44fe3-9676-450b-a7ee-2754cd436d84 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:36.926476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:36.927276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.927732 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.928170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.928991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:36.929454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:36.931263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:36.948489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 892/1782] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:43:36.949445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:36.959542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150c03e-09b6-40e4-84b1-5e91a2025bc9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:36.965041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:36.965804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:36.966169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:36.966516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:36.967224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:36.969257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.003491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 891/1783] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:36 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:37.004704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.018113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca4bb0-17d3-4130-b95c-21aa7817157a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.021232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.023501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.069545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:37.090554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.091194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.091509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.091781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.092416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.095469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.098259 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7631-4ad9-424c-b048-55cd0db6f8ee admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.104496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.105217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.105994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.106346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] PATH_INFO: `/v3/users/e4ebebb143424d6bb500909951ce87ce` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.109346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ebebb143424d6bb500909951ce87ce)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.111460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.123824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:37.129399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 893/1784] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:37 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:37.130798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.144426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e018ba-e521-48e0-a86d-1a2bbd1c6a7b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.150847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.152177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.154430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.154430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.154430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.155279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.171083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 894/1785] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:43:37.175916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.189571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a62dc-bc13-43d7-bea6-49de82c83d0c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.192035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ebebb143424d6bb500909951ce87ce'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.192974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ebebb143424d6bb500909951ce87ce'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.198648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.200197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.200478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.200753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.201411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.203499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ebebb143424d6bb500909951ce87ce'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.203669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.204440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ebebb143424d6bb500909951ce87ce'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.216822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 895/1786] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:37.217832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bbf41148-ad7b-4dfa-8794-d848637fca38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ebebb143424d6bb500909951ce87ce'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.221059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.222766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.236695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a46ee-b852-45b9-a1e1-26ac11b8646c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.241439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fd24a-b026-4b87-9c15-3f0a54fb0ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.242103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fd24a-b026-4b87-9c15-3f0a54fb0ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.242382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fd24a-b026-4b87-9c15-3f0a54fb0ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.242655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fd24a-b026-4b87-9c15-3f0a54fb0ad7 admin admin] PATH_INFO: `/v3/projects/b67d4b040bc94fd382ace03f7b858275/users/b065dc5a7dba4c869a2b64c82f5cc1c7/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.243770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 892/1787] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:37 2022] DELETE /identity/v3/users/e4ebebb143424d6bb500909951ce87ce => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:37.249168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.251782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.252134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.261990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fd24a-b026-4b87-9c15-3f0a54fb0ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67d4b040bc94fd382ace03f7b858275, user_id=b065dc5a7dba4c869a2b64c82f5cc1c7, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.264810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fd24a-b026-4b87-9c15-3f0a54fb0ad7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.277619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:37.278934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.281354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 896/1788] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:37 2022] PUT /identity/v3/projects/b67d4b040bc94fd382ace03f7b858275/users/b065dc5a7dba4c869a2b64c82f5cc1c7/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:43:37.294467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.295483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.295836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.324299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] MFA Rules not processed for user `b065dc5a7dba4c869a2b64c82f5cc1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:37.325513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.378970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 893/1789] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:37.392029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.394390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.405344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 897/1790] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:37.417520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.418950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.466622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098001da-9c10-44bb-bc14-f8ddf266468a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.473333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.474180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.474462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.474734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.484613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.485485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9bcb-a7eb-40fe-8cb7-db3cf20c8732 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.486675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.489159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.489771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.490053 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.490325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.497435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 894/1791] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:43:37.500455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.501882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.511691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 898/1792] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:43:37.835860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.837349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:37.851769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4734ce-3baf-433f-a71a-ae63008847e0 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:37.860447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:37.861755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:37.865692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:37.865692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] PATH_INFO: `/v3/projects/f5b75b666c884d09851cca0c0730a1f0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:37.867729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b75b666c884d09851cca0c0730a1f0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:37.870336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:37.927891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:37.949244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b75b666c884d09851cca0c0730a1f0', 'request_id': 'req-d034ca52-e525-4897-a161-28f80c2664e7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.955729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b75b666c884d09851cca0c0730a1f0', 'request_id': 'req-d034ca52-e525-4897-a161-28f80c2664e7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.963769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b75b666c884d09851cca0c0730a1f0', 'request_id': 'req-d034ca52-e525-4897-a161-28f80c2664e7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.974163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] The token cache is being invalidate because project f5b75b666c884d09851cca0c0730a1f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:37.974448 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.975078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:37.976604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 895/1793] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:37 2022] DELETE /identity/v3/projects/f5b75b666c884d09851cca0c0730a1f0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:37.981141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:37.991994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91ec26-8268-418f-b161-8f2b66e2fed0 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:38.008520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:38.048771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:38.049394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:38.049675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:38.049950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] PATH_INFO: `/v3/users/73473a9d52214d34b58606c1fcfacb61` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:38.051054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73473a9d52214d34b58606c1fcfacb61)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:38.053022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:38.098119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73473a9d52214d34b58606c1fcfacb61'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.098766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73473a9d52214d34b58606c1fcfacb61'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.099367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73473a9d52214d34b58606c1fcfacb61'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.106828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73473a9d52214d34b58606c1fcfacb61'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.113880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73473a9d52214d34b58606c1fcfacb61'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.131000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 899/1794] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:37 2022] DELETE /identity/v3/users/73473a9d52214d34b58606c1fcfacb61 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:38.418876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:38.421066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:38.447132 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ca52-e525-4897-a161-28f80c2664e7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:38.524518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:38.525260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:38.526160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:38.526544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] PATH_INFO: `/v3/projects/dd67fa0b8c34463c8e0ca7d9ae3cb1e0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:38.527781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd67fa0b8c34463c8e0ca7d9ae3cb1e0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:38.530061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:38.584896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd67fa0b8c34463c8e0ca7d9ae3cb1e0', 'request_id': 'req-5dae27f6-ddef-4492-b079-3c785c0c3bb9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.594231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd67fa0b8c34463c8e0ca7d9ae3cb1e0', 'request_id': 'req-5dae27f6-ddef-4492-b079-3c785c0c3bb9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.603760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd67fa0b8c34463c8e0ca7d9ae3cb1e0', 'request_id': 'req-5dae27f6-ddef-4492-b079-3c785c0c3bb9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.615290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] The token cache is being invalidate because project dd67fa0b8c34463c8e0ca7d9ae3cb1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:38.615614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.616317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.617990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 896/1795] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:38 2022] DELETE /identity/v3/projects/dd67fa0b8c34463c8e0ca7d9ae3cb1e0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:38.621908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:38.624577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:38.637839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612ce8-59a6-4b56-a4dd-d4430c6bec23 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:38.707899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:38.708610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:38.709143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:38.709519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] PATH_INFO: `/v3/users/6472fc3e67374873a44f193e9a6e4983` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:38.710811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6472fc3e67374873a44f193e9a6e4983)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:38.713763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:38.759357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:38.767080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6472fc3e67374873a44f193e9a6e4983'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.767815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6472fc3e67374873a44f193e9a6e4983'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.768585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6472fc3e67374873a44f193e9a6e4983'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.776294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6472fc3e67374873a44f193e9a6e4983'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.783901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d449de4f-c7e9-4f3b-8d7d-4db0d3734780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6472fc3e67374873a44f193e9a6e4983'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:38.801305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 900/1796] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:38 2022] DELETE /identity/v3/users/6472fc3e67374873a44f193e9a6e4983 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:39.136240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.138385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:39.158883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae27f6-ddef-4492-b079-3c785c0c3bb9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.223435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:39.241690 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.242655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.243126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.243620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] PATH_INFO: `/v3/projects/9295b06bb80741ac849cdb2ed0c2f30f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.245479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9295b06bb80741ac849cdb2ed0c2f30f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.247846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.295245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9295b06bb80741ac849cdb2ed0c2f30f', 'request_id': 'req-2960d289-4f0e-4093-a2df-b5dcf2a41a04'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:39.301457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9295b06bb80741ac849cdb2ed0c2f30f', 'request_id': 'req-2960d289-4f0e-4093-a2df-b5dcf2a41a04'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:39.309581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9295b06bb80741ac849cdb2ed0c2f30f', 'request_id': 'req-2960d289-4f0e-4093-a2df-b5dcf2a41a04'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:39.320404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] The token cache is being invalidate because project 9295b06bb80741ac849cdb2ed0c2f30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:39.320692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:39.321309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2960d289-4f0e-4093-a2df-b5dcf2a41a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:39.322828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 897/1797] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:39 2022] DELETE /identity/v3/projects/9295b06bb80741ac849cdb2ed0c2f30f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:39.328831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.329310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.329420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.349681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:39.350876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:39.408476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 901/1798] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:39.418723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.419328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.419636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.439711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:39.440914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:39.448993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 898/1799] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:39.453156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.463348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c06c-e2ff-4818-a751-aa1096c329da None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.468000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.468765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.469097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.469372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.470078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:39.470667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.472294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.484594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 902/1800] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:43:39.489072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.499037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213d7ce-b91c-4ce9-b0de-19737183c70e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.503850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.504457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.504734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.505005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.505666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.507677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.537652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 899/1801] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:39 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:39.538797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.549449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d07c7-8457-4e0b-b92e-6c5fef8999e1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.604600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.605194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.605469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.605735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.606348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.608269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.632427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 903/1802] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:39 2022] POST /identity/v3/users => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:39.636580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.647197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0d940-babe-48ec-a173-fb335649dc2b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.654644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.655284 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.655598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.655874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.656531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.658166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.671039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 900/1803] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:39.674984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.685623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce428b21-c5f5-4605-b110-da1d02accf1b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.690523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22c78e-7cb9-4865-86db-bee7402dbb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.691150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c78e-7cb9-4865-86db-bee7402dbb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.691446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c78e-7cb9-4865-86db-bee7402dbb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.691766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c78e-7cb9-4865-86db-bee7402dbb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.692477 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22c78e-7cb9-4865-86db-bee7402dbb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.694232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22c78e-7cb9-4865-86db-bee7402dbb99 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.708913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 904/1804] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:43:39.710272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.721288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bedc30-eed7-4569-87f3-9b154e6209d4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.726095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.726772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.727054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.727329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] PATH_INFO: `/v3/projects/e9a63c61b4f44604ba75d73128d463c8/users/b93c62fea26b436ab5dc5fbb85f5aaa5/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.745152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a63c61b4f44604ba75d73128d463c8, user_id=b93c62fea26b436ab5dc5fbb85f5aaa5, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:39.750073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:39.765035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 901/1805] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:39 2022] PUT /identity/v3/projects/e9a63c61b4f44604ba75d73128d463c8/users/b93c62fea26b436ab5dc5fbb85f5aaa5/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:39.777793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.779303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.779621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.810860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] MFA Rules not processed for user `b93c62fea26b436ab5dc5fbb85f5aaa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:39.812137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:39.825087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:39.882485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 905/1806] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:39.920057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:39.922073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:39.985567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b02b7-6a32-455e-945b-eea45fef9b73 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:39.989303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b22f0-4e74-4405-bd37-00e7daa195e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:39.989951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b22f0-4e74-4405-bd37-00e7daa195e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:39.990167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b22f0-4e74-4405-bd37-00e7daa195e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:39.990433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b22f0-4e74-4405-bd37-00e7daa195e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:39.998837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b22f0-4e74-4405-bd37-00e7daa195e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:40.000325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b22f0-4e74-4405-bd37-00e7daa195e5 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:40.008983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 902/1807] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:40.025192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:40.086236 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820e35e-5741-4be1-b633-efbc35812cf1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:40.091483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:40.092109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.092389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.092663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] PATH_INFO: `/v3/users/b93c62fea26b436ab5dc5fbb85f5aaa5` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.093794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93c62fea26b436ab5dc5fbb85f5aaa5)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:40.095829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:40.142128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93c62fea26b436ab5dc5fbb85f5aaa5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.142871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93c62fea26b436ab5dc5fbb85f5aaa5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.143645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93c62fea26b436ab5dc5fbb85f5aaa5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.151727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93c62fea26b436ab5dc5fbb85f5aaa5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.159850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93c62fea26b436ab5dc5fbb85f5aaa5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.178168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 906/1808] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:40 2022] DELETE /identity/v3/users/b93c62fea26b436ab5dc5fbb85f5aaa5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:40.183275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.183786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.183970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.204325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:40.205531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:40.214637 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:40.275175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 903/1809] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:40.285299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:40.286786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:40.324261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:40.348215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c92f2-a52a-4845-a817-893059b03974 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:40.351832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6900ef-2c95-4088-bb44-8f05a130164f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:40.352436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6900ef-2c95-4088-bb44-8f05a130164f service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.352715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6900ef-2c95-4088-bb44-8f05a130164f service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.352985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6900ef-2c95-4088-bb44-8f05a130164f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.362518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6900ef-2c95-4088-bb44-8f05a130164f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:40.363969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6900ef-2c95-4088-bb44-8f05a130164f service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:40.374885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 907/1810] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:43:40.731746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:40.733336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:40.748472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f1b51-d9cc-44b4-a8fc-c20dcc601bec None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:40.753912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:40.755088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.755358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.755760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] PATH_INFO: `/v3/projects/e9a63c61b4f44604ba75d73128d463c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.756820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a63c61b4f44604ba75d73128d463c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:40.759013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:40.805862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a63c61b4f44604ba75d73128d463c8', 'request_id': 'req-7fc194a5-14f5-4e8d-a846-2a33c7add100'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.813806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a63c61b4f44604ba75d73128d463c8', 'request_id': 'req-7fc194a5-14f5-4e8d-a846-2a33c7add100'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.821604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a63c61b4f44604ba75d73128d463c8', 'request_id': 'req-7fc194a5-14f5-4e8d-a846-2a33c7add100'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.831978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] The token cache is being invalidate because project e9a63c61b4f44604ba75d73128d463c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:40.832268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.832899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7fc194a5-14f5-4e8d-a846-2a33c7add100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:40.834439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 904/1811] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:40 2022] DELETE /identity/v3/projects/e9a63c61b4f44604ba75d73128d463c8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:40.839353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.839671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.839947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.859368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:40.860596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:40.919096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 908/1812] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:40.926757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.927001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.927268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.947733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:40.948919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:40.957002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 905/1813] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:40.960828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:40.970241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeb0b5-07c8-44cb-b8b7-86eaded77feb None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:40.975041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:40.975691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:40.975970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:40.976243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:40.976899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:40.977202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:40.979476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:40.991966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 909/1814] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:43:40.996846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.006565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db6bbb-5c69-4cb8-b224-4b0f18188383 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.012171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.012894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.013170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.013442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.014084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.016127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.047483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 906/1815] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:40 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:41.050032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.060210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52abdff-41de-4ee5-b84d-82e7499b77f5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.118568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.119106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.119388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.119692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.120333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.122247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.142633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 910/1816] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:41.146765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.157747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ad6cf-903d-4f6c-96dc-6f63e048c13c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.162319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.162911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.163184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.163474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.164109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.165694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.176388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 907/1817] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:41.180327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.190066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3c2a7-712f-4568-aa87-9d6f394efa82 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.194752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5046-c8cc-4bd5-9e71-b0b629831dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.195351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5046-c8cc-4bd5-9e71-b0b629831dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.195681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5046-c8cc-4bd5-9e71-b0b629831dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.195954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5046-c8cc-4bd5-9e71-b0b629831dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.196596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5046-c8cc-4bd5-9e71-b0b629831dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.198208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5046-c8cc-4bd5-9e71-b0b629831dfd admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.210455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 911/1818] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:41.215122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.225262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee23cc-259e-4c12-9043-559b7cd7558b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.230402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.231040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.231677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.231951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] PATH_INFO: `/v3/projects/0063db7d04e44532b50f638d19d92aa4/users/55ff0296c65645948b53df4782521e7c/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.246219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063db7d04e44532b50f638d19d92aa4, user_id=55ff0296c65645948b53df4782521e7c, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.248908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.261603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 908/1819] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:41 2022] PUT /identity/v3/projects/0063db7d04e44532b50f638d19d92aa4/users/55ff0296c65645948b53df4782521e7c/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:41.272620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.273055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.273338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.303814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] MFA Rules not processed for user `55ff0296c65645948b53df4782521e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:41.305018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:41.333242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:41.367755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 912/1820] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:41.372295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.417751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:41.448149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815a2a-54d7-4a92-91de-caf8af739823 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.452805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.453405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.453685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.453959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.454604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.456556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.482889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 909/1821] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:41.486902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.497572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f185e1-f9db-4d28-95bd-b6c7e02a4345 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.553091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.553696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.553976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.554248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.554874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.556872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.579892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 913/1822] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:41.583774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.594927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb989d-b1ad-4e28-9feb-8013ac2c080c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.599679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.600277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.600552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.600822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.601458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.603034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.614644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 910/1823] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:41.619051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.629184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bafdbf-22d5-445d-8728-933b98631c71 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.633848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e0197-48e4-4120-ad3a-fa6ea0ac4c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.634453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e0197-48e4-4120-ad3a-fa6ea0ac4c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.634734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e0197-48e4-4120-ad3a-fa6ea0ac4c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.635539 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e0197-48e4-4120-ad3a-fa6ea0ac4c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.636190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e0197-48e4-4120-ad3a-fa6ea0ac4c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.637773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e0197-48e4-4120-ad3a-fa6ea0ac4c44 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.647833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 914/1824] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:41.652658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.662894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36574672-002b-4e87-a4e2-c77840426199 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.667763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.668419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.668697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.668969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] PATH_INFO: `/v3/projects/d630d51db77d4dc09ac68a274cae0685/users/175d87ba442443fb8344319afb4c0910/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.683684 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d630d51db77d4dc09ac68a274cae0685, user_id=175d87ba442443fb8344319afb4c0910, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.686387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.701484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 911/1825] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:41 2022] PUT /identity/v3/projects/d630d51db77d4dc09ac68a274cae0685/users/175d87ba442443fb8344319afb4c0910/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:41.711647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.711937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.712212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.741265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] MFA Rules not processed for user `175d87ba442443fb8344319afb4c0910`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:41.742464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:41.806473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 915/1826] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:41.811737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.871698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cd613-1b85-423a-bdca-9e70b6eff151 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.876230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.876821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.877096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.877365 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.877993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.879963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:41.905872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 912/1827] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:41.910023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:41.920801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd1d18-0567-4b4b-bc17-f28568f71681 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:41.958559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:41.976982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:41.977579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:41.977851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:41.978117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:41.978777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:41.980716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:42.000956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 916/1828] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:41 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:42.005103 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:42.015835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae896183-fc8c-4627-897c-7c630bbd7206 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:42.020308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:42.020922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:42.021198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:42.021465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:42.022093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:42.023709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:42.034241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 913/1829] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:42.038187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:42.047797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad81b97-9509-434a-bdcf-ebc7ddabf266 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:42.052286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:42.052922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:42.053197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:42.053464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] PATH_INFO: `/v3/projects/7810ad03ea6143c08224c4e6fd6428f0/users/9c7937941c6c44deb2b2aa6b75959add/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:42.068477 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7810ad03ea6143c08224c4e6fd6428f0, user_id=9c7937941c6c44deb2b2aa6b75959add, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:42.071240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:42.084096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 917/1830] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:42 2022] PUT /identity/v3/projects/7810ad03ea6143c08224c4e6fd6428f0/users/9c7937941c6c44deb2b2aa6b75959add/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:42.088116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:42.098116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22350fcf-48e6-453b-a0d3-e297f4b13ffc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:42.149548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:42.157538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fec2da-664a-452c-8de6-cebf01f10218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:42.158154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fec2da-664a-452c-8de6-cebf01f10218 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:42.158435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fec2da-664a-452c-8de6-cebf01f10218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:42.158710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fec2da-664a-452c-8de6-cebf01f10218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:42.159360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fec2da-664a-452c-8de6-cebf01f10218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:42.160996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fec2da-664a-452c-8de6-cebf01f10218 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:42.171975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 914/1831] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:42.176159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:42.187025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6910d-8822-40ed-bf55-b74273390f5f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:42.192296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:42.192982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:42.193312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:42.193585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] PATH_INFO: `/v3/projects/7810ad03ea6143c08224c4e6fd6428f0/users/9c7937941c6c44deb2b2aa6b75959add/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:42.195896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7810ad03ea6143c08224c4e6fd6428f0, user_id=9c7937941c6c44deb2b2aa6b75959add, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:42.198949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:42.210302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 918/1832] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:42 2022] PUT /identity/v3/projects/7810ad03ea6143c08224c4e6fd6428f0/users/9c7937941c6c44deb2b2aa6b75959add/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:42.221566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:42.222205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:42.222205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:42.250807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] MFA Rules not processed for user `9c7937941c6c44deb2b2aa6b75959add`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:42.252069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:42.323533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 915/1833] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:42.334790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:42.336506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:42.392998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69fb92-2350-4c4f-9981-94e741bbca00 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:42.396681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:42.397272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:42.397549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:42.397818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:42.453071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:42.454840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:42.463254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 919/1834] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:44.003366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:44.005070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:44.017593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abd637-349d-435d-8db2-4313e1d276c3 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:44.021482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:44.022028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:44.022309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:44.022577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:44.083600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:44.085057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:44.095498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 916/1835] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:44.685602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:44.694380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e218bd-2984-49b3-bfad-5f13135909d3 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:44.698621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6e8f3-73fb-49c3-8825-49c395106b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:44.699221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6e8f3-73fb-49c3-8825-49c395106b13 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:44.699530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6e8f3-73fb-49c3-8825-49c395106b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:44.699831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6e8f3-73fb-49c3-8825-49c395106b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:44.708857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6e8f3-73fb-49c3-8825-49c395106b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:44.710269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6e8f3-73fb-49c3-8825-49c395106b13 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:44.720646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 920/1836] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:43:46.227688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:46.229492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:46.299568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8980ad-a526-4850-82f6-eaa60efbdd57 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:46.303982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:46.304599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:46.304878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:46.305144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] PATH_INFO: `/v3/users/b065dc5a7dba4c869a2b64c82f5cc1c7` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:46.306270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b065dc5a7dba4c869a2b64c82f5cc1c7)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:46.308746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:46.329588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:46.354017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b065dc5a7dba4c869a2b64c82f5cc1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:46.354653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b065dc5a7dba4c869a2b64c82f5cc1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:46.361792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b065dc5a7dba4c869a2b64c82f5cc1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:46.362882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b065dc5a7dba4c869a2b64c82f5cc1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:46.370754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b065dc5a7dba4c869a2b64c82f5cc1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:46.390522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 917/1837] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:46 2022] DELETE /identity/v3/users/b065dc5a7dba4c869a2b64c82f5cc1c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:46.394809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:46.395292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:46.395384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:46.425077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:46.426296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:46.457870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:46.495335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 921/1838] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:46.503307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:46.504904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:46.577075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44bb1-2ff5-4ec2-b135-5670e5376c3e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:46.580831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3829-fc49-4f94-bccb-eb93138866dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:46.581428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3829-fc49-4f94-bccb-eb93138866dd service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:46.581710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3829-fc49-4f94-bccb-eb93138866dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:46.581979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3829-fc49-4f94-bccb-eb93138866dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:46.590878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3829-fc49-4f94-bccb-eb93138866dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:46.592328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3829-fc49-4f94-bccb-eb93138866dd service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:46.602089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 918/1839] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:47.115380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.117027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:47.129258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8ac3-2b55-4038-9b3b-86e7f20e80a0 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.134633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.135278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.135585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.135902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] PATH_INFO: `/v3/projects/b67d4b040bc94fd382ace03f7b858275` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.136982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67d4b040bc94fd382ace03f7b858275)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.138884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.186082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67d4b040bc94fd382ace03f7b858275', 'request_id': 'req-b9d6f3e6-6085-429a-aa3b-5eefd0883002'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:47.195867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67d4b040bc94fd382ace03f7b858275', 'request_id': 'req-b9d6f3e6-6085-429a-aa3b-5eefd0883002'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:47.202383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67d4b040bc94fd382ace03f7b858275', 'request_id': 'req-b9d6f3e6-6085-429a-aa3b-5eefd0883002'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:47.213837 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] The token cache is being invalidate because project b67d4b040bc94fd382ace03f7b858275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:47.214204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:47.215019 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b9d6f3e6-6085-429a-aa3b-5eefd0883002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:47.216893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 922/1840] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:47 2022] DELETE /identity/v3/projects/b67d4b040bc94fd382ace03f7b858275 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:47.223890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.224188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.224463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.251610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:47.252854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:47.333238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 919/1841] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:47.335591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.335884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.336156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.379962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:47.379962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:47.381175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 923/1842] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:47.386436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.399705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62385f8b-ac31-43c5-b36d-1b36901a76b7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.419366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.420811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.421187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.421571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.423265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:47.424875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.427474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.442068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 920/1843] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:43:47.446810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.458110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbeb14e-6dda-4489-8dad-fc85dab82152 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.463818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.464435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.464713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.465065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.465736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.467900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.518773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 924/1844] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:47 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:47.521482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.533759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a2539-a055-4f3d-bd85-8e95feec9378 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.564498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:47.642682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.644056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.644440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.645065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.645641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.650987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.679191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 921/1845] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:47 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:47.683902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.699356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948749d0-2b1f-44a8-ab32-40cd6bc4642d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.714746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.716128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.716411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.716850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.717694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.721823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.746514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 925/1846] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:47.751462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.767850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ca16-2654-41c9-a5c5-b8b27d2ee524 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.779656 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebfb3-b24d-4b36-ada6-eeeb2c4c2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.780570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebfb3-b24d-4b36-ada6-eeeb2c4c2734 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.780898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebfb3-b24d-4b36-ada6-eeeb2c4c2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.781403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebfb3-b24d-4b36-ada6-eeeb2c4c2734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.782318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ebfb3-b24d-4b36-ada6-eeeb2c4c2734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.784194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ebfb3-b24d-4b36-ada6-eeeb2c4c2734 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.798623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 922/1847] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:43:47.805098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:47.817032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14064497-1a5b-43d5-b114-e402248bebe7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:47.829675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:47.830670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.831243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.831653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] PATH_INFO: `/v3/projects/15f10d31602e42fa9d43d091e8d0c80f/users/d5f5e29de7c84f8c8635730f26e357fc/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.848750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f10d31602e42fa9d43d091e8d0c80f, user_id=d5f5e29de7c84f8c8635730f26e357fc, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:47.851538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:47.867818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 926/1848] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:47 2022] PUT /identity/v3/projects/15f10d31602e42fa9d43d091e8d0c80f/users/d5f5e29de7c84f8c8635730f26e357fc/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:47.879252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:47.879571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:47.879846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:47.927473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] MFA Rules not processed for user `d5f5e29de7c84f8c8635730f26e357fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:47.928664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:47.994747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 923/1849] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:48.005927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:48.007673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:48.063790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3894dd-9c4b-42f3-b634-dc023769ca16 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:48.067466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9912b-8f04-48a3-939d-5a295a20da5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:48.068073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9912b-8f04-48a3-939d-5a295a20da5e service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:48.068354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9912b-8f04-48a3-939d-5a295a20da5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:48.068626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9912b-8f04-48a3-939d-5a295a20da5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:48.075767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:48.078803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9912b-8f04-48a3-939d-5a295a20da5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:48.080638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9912b-8f04-48a3-939d-5a295a20da5e service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:48.099563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 927/1850] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:49.012474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:49.014867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:49.117602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed42f0-ca99-47f6-b9b3-91342023bb9a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:49.124463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:49.125118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:49.125418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:49.125684 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] PATH_INFO: `/v3/users/55ff0296c65645948b53df4782521e7c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:49.126929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ff0296c65645948b53df4782521e7c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:49.131127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:49.253839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:49.266874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ff0296c65645948b53df4782521e7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.268117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ff0296c65645948b53df4782521e7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.279155 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ff0296c65645948b53df4782521e7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.280511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ff0296c65645948b53df4782521e7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.290016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ff0296c65645948b53df4782521e7c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.316908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 924/1851] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:49 2022] DELETE /identity/v3/users/55ff0296c65645948b53df4782521e7c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:49.321338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:49.321791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:49.321921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:49.342421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:49.344089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:49.407317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 928/1852] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:49.419330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:49.421066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:49.478133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ca66e-7f93-45b1-8d59-c93586c93939 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:49.482685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:49.483320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:49.483624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:49.484027 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:49.493527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:49.494993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:49.504575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 925/1853] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:49.870547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:49.872051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:49.887150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839a442-357d-4c7d-a568-f64b452568ca None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:49.892688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:49.893451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:49.893757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:49.894000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] PATH_INFO: `/v3/projects/0063db7d04e44532b50f638d19d92aa4` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:49.895595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0063db7d04e44532b50f638d19d92aa4)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:49.900097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:49.955816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0063db7d04e44532b50f638d19d92aa4', 'request_id': 'req-c0e0114e-d071-4fd0-ac5e-6f4dce725946'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.964054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0063db7d04e44532b50f638d19d92aa4', 'request_id': 'req-c0e0114e-d071-4fd0-ac5e-6f4dce725946'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.970947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0063db7d04e44532b50f638d19d92aa4', 'request_id': 'req-c0e0114e-d071-4fd0-ac5e-6f4dce725946'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.983310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] The token cache is being invalidate because project 0063db7d04e44532b50f638d19d92aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:49.983833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.984703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:49.986446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 929/1854] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:49 2022] DELETE /identity/v3/projects/0063db7d04e44532b50f638d19d92aa4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:49.990665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:50.001653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b6905-b7f6-49b2-a7b4-5bb8c0d9eafd service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:50.077234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:50.077878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:50.078161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:50.078436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] PATH_INFO: `/v3/users/175d87ba442443fb8344319afb4c0910` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:50.079582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=175d87ba442443fb8344319afb4c0910)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:50.081542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:50.129955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175d87ba442443fb8344319afb4c0910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.130559 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175d87ba442443fb8344319afb4c0910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.138351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175d87ba442443fb8344319afb4c0910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.138991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175d87ba442443fb8344319afb4c0910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.142334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:50.148795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175d87ba442443fb8344319afb4c0910'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.168874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 926/1855] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:49 2022] DELETE /identity/v3/users/175d87ba442443fb8344319afb4c0910 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:43:50.498286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:50.499812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:50.512438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0114e-d071-4fd0-ac5e-6f4dce725946 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:50.573227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:50.573865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:50.574144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:50.574416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] PATH_INFO: `/v3/projects/d630d51db77d4dc09ac68a274cae0685` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:50.575584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d630d51db77d4dc09ac68a274cae0685)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:50.577490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:50.600302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:50.625030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd630d51db77d4dc09ac68a274cae0685', 'request_id': 'req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.631861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd630d51db77d4dc09ac68a274cae0685', 'request_id': 'req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.638072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd630d51db77d4dc09ac68a274cae0685', 'request_id': 'req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.648292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] The token cache is being invalidate because project d630d51db77d4dc09ac68a274cae0685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:50.648574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.649218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.651008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 930/1856] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:50 2022] DELETE /identity/v3/projects/d630d51db77d4dc09ac68a274cae0685 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:50.655218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:50.657176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:50.673298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:50.674802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:50.675465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d900-3352-4ed5-8eb3-e29c74ef6bd1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:50.736640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:50.737293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:50.737573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:50.737845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] PATH_INFO: `/v3/users/9c7937941c6c44deb2b2aa6b75959add` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:50.740418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c7937941c6c44deb2b2aa6b75959add)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:50.743736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:50.745654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e33ab-b8cd-45d0-b785-b1111a9d0b08 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:50.751111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:50.752196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:50.753467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:50.753740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] PATH_INFO: `/v3/users/d5f5e29de7c84f8c8635730f26e357fc` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:50.754908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f5e29de7c84f8c8635730f26e357fc)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:50.756892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:50.797984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c7937941c6c44deb2b2aa6b75959add'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.798612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c7937941c6c44deb2b2aa6b75959add'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.805703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c7937941c6c44deb2b2aa6b75959add'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.806511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c7937941c6c44deb2b2aa6b75959add'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.811998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f5e29de7c84f8c8635730f26e357fc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.812749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f5e29de7c84f8c8635730f26e357fc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.813464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f5e29de7c84f8c8635730f26e357fc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.813666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6d0af5e4-c670-4876-af6c-71572d989e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c7937941c6c44deb2b2aa6b75959add'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.820634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f5e29de7c84f8c8635730f26e357fc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.829242 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f5e29de7c84f8c8635730f26e357fc'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:50.831493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 927/1857] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:50 2022] DELETE /identity/v3/users/9c7937941c6c44deb2b2aa6b75959add => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:50.855981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 931/1858] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:43:50 2022] DELETE /identity/v3/users/d5f5e29de7c84f8c8635730f26e357fc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:43:50.860285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:50.860577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:50.860857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:50.893947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:50.895282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:50.967868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:50.985650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 928/1859] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:50.995897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.000324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.038430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:51.068003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea407d6a-8abe-4471-b24f-2c94d954d37d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.072733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d4662-f501-4b06-9104-2dd8d8259bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.073344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4662-f501-4b06-9104-2dd8d8259bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.073619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4662-f501-4b06-9104-2dd8d8259bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.073944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4662-f501-4b06-9104-2dd8d8259bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.085571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d4662-f501-4b06-9104-2dd8d8259bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.087132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d4662-f501-4b06-9104-2dd8d8259bf9 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.100230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 932/1860] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:51.189328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.191327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.223752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1381b2-ce10-4cef-a213-6fd1d7b6ab5a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.232388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.233139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.233513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.234436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] PATH_INFO: `/v3/projects/7810ad03ea6143c08224c4e6fd6428f0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.235471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7810ad03ea6143c08224c4e6fd6428f0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.237474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.296922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7810ad03ea6143c08224c4e6fd6428f0', 'request_id': 'req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.305639 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7810ad03ea6143c08224c4e6fd6428f0', 'request_id': 'req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.315242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7810ad03ea6143c08224c4e6fd6428f0', 'request_id': 'req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.328981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] The token cache is being invalidate because project 7810ad03ea6143c08224c4e6fd6428f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:51.329282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.330161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.332630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 929/1861] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:51 2022] DELETE /identity/v3/projects/7810ad03ea6143c08224c4e6fd6428f0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:51.338084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c112-106d-4a89-819c-c5b3c69b2ca2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.338491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c112-106d-4a89-819c-c5b3c69b2ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.338831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c112-106d-4a89-819c-c5b3c69b2ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.372311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e494c112-106d-4a89-819c-c5b3c69b2ca2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:51.385923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e494c112-106d-4a89-819c-c5b3c69b2ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.442540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.446090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.478714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 933/1862] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:51.486991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.487277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.487637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.514635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:51.515952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.534504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 934/1863] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:51.538720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.549091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312bd7-8298-4e9e-87c9-9d458d12d0f7 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.554379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.554993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.555271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.555575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.555711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21dc1e-773a-40da-8f9b-077ebff5ef4c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.556739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:51.557212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.558806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.562408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.563052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.563334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.563647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] PATH_INFO: `/v3/projects/15f10d31602e42fa9d43d091e8d0c80f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.564800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f10d31602e42fa9d43d091e8d0c80f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.566742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.575085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 935/1864] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:43:51.575860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.585935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135fd92-506a-4e50-8ecb-29117dc66442 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.590535 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.591138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.591440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.591718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.592343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.594850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.630826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f10d31602e42fa9d43d091e8d0c80f', 'request_id': 'req-68426219-ab28-4f8b-ba47-a992b65130c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.631334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 936/1865] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:51.637698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.641776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f10d31602e42fa9d43d091e8d0c80f', 'request_id': 'req-68426219-ab28-4f8b-ba47-a992b65130c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.652776 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f10d31602e42fa9d43d091e8d0c80f', 'request_id': 'req-68426219-ab28-4f8b-ba47-a992b65130c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.669213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] The token cache is being invalidate because project 15f10d31602e42fa9d43d091e8d0c80f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:43:51.669615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.670437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-68426219-ab28-4f8b-ba47-a992b65130c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:43:51.675666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 930/1866] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:43:51 2022] DELETE /identity/v3/projects/15f10d31602e42fa9d43d091e8d0c80f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:51.677475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e536ccd-231e-4376-99c9-13092f9e4d6f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.677735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e536ccd-231e-4376-99c9-13092f9e4d6f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.678007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e536ccd-231e-4376-99c9-13092f9e4d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.704283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7e536ccd-231e-4376-99c9-13092f9e4d6f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:51.705589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7e536ccd-231e-4376-99c9-13092f9e4d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.710843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:51.736014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03437c-ee8c-45c5-aa50-827c566a5f03 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.742653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.743529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.744527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.744871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.745582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.747820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.773795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 937/1867] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:51.778487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.779658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 931/1868] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:51.780579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.787646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.787930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.788488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.794131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0d252-b981-4d2f-973d-698f809a4b73 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.799792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.800405 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.800687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.801013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.801727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.803475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.818666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 938/1869] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:51.819553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.820126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:51.821859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:51.838043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 932/1870] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:51.842137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.845757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc755a-6b29-484a-bb7d-9ab53818c178 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.853771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.854596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d900d-af67-4639-9e8b-15f522f4762e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.854752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.854869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.855158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.857367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.858908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.861009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.861862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.862192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.862527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.863205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:43:51.863338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:51.863541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.865371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.878971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 939/1871] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 06:43:51.879581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 933/1872] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 06:43:51.880508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.882403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.893457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c74889-de71-4419-a13c-7ba3807241a3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.897576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f7c0e-bf48-4cc4-90be-f0359e2c3fe5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:51.899093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.899738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.900630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.900900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.901543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.903583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.905240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241cabbc-87da-4bf6-a4ef-334412984d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:51.906937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cabbc-87da-4bf6-a4ef-334412984d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.908109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cabbc-87da-4bf6-a4ef-334412984d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.908877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cabbc-87da-4bf6-a4ef-334412984d28 admin admin] PATH_INFO: `/v3/projects/d691da1e1a4045dc9255e1dd61d248b3/users/405122a8524e4ab78cc7c3c260f5ec5f/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.929533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241cabbc-87da-4bf6-a4ef-334412984d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d691da1e1a4045dc9255e1dd61d248b3, user_id=405122a8524e4ab78cc7c3c260f5ec5f, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:51.933953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241cabbc-87da-4bf6-a4ef-334412984d28 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:51.938865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 934/1873] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:43:51.940333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:51.950221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 940/1874] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:51 2022] PUT /identity/v3/projects/d691da1e1a4045dc9255e1dd61d248b3/users/405122a8524e4ab78cc7c3c260f5ec5f/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:51.960853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:51.961196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:51.961411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:51.997471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] MFA Rules not processed for user `405122a8524e4ab78cc7c3c260f5ec5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:51.999008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:52.021354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a2e22-8e2b-4189-9133-356992e10a8b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:52.085404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 941/1875] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:52.090159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:52.091021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.091331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.092034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.092870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:52.095192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:52.107698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:52.107698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:52.119782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 935/1876] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:43:51 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:43:52.124747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:52.131308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:52.136142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14a0d-7f3b-4590-97fd-e48d74f55132 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:52.142480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:52.142953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.143364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.143671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.144324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:52.146204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:52.161735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 936/1877] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:52.162627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:52.176400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce856ce-2858-45f8-bf42-ac2f645a46a0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:52.183630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:52.184728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.185197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.186082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.187055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:52.189109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:52.203792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 937/1878] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:43:52.208593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f23d37-f1d1-48a0-8abc-a86869ef6e2d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:52.208747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:52.213599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f841c1f-db60-49e8-aafe-38d9a052f7ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:52.214490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841c1f-db60-49e8-aafe-38d9a052f7ab service glance] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.215086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841c1f-db60-49e8-aafe-38d9a052f7ab service glance] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.215745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841c1f-db60-49e8-aafe-38d9a052f7ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.223090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61a769-8859-4e9c-9e4b-cebad0e4292b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:52.226853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f841c1f-db60-49e8-aafe-38d9a052f7ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:52.228619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f841c1f-db60-49e8-aafe-38d9a052f7ab service glance] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:52.228847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:52.229525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.229804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.230080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] PATH_INFO: `/v3/projects/9ca42ed577ae40258f333875e0efbd62/users/cccdf0e0e8214d11867cc9a47c997b40/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.241971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 942/1879] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:52.251349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca42ed577ae40258f333875e0efbd62, user_id=cccdf0e0e8214d11867cc9a47c997b40, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:52.255533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:52.273039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 938/1880] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:43:52 2022] PUT /identity/v3/projects/9ca42ed577ae40258f333875e0efbd62/users/cccdf0e0e8214d11867cc9a47c997b40/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:43:52.285148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90e979-9098-4441-bff8-904a265cd48e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.285436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90e979-9098-4441-bff8-904a265cd48e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.285700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90e979-9098-4441-bff8-904a265cd48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.325096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3b90e979-9098-4441-bff8-904a265cd48e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:52.326534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3b90e979-9098-4441-bff8-904a265cd48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:52.401319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:52.444796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 943/1881] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:52.456521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:52.457789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:52.556650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67f9f-c2a5-4b89-892c-d9f970bb0a27 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:52.563950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:52.564828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:52.565115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:52.565386 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:52.578867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:52.581056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:52.595693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 939/1882] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:53.783015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50a1c-8435-4c42-8f42-c88bf1e557fb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:53.783984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50a1c-8435-4c42-8f42-c88bf1e557fb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:53.784626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50a1c-8435-4c42-8f42-c88bf1e557fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:53.817748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8da50a1c-8435-4c42-8f42-c88bf1e557fb None None] MFA Rules not processed for user `cccdf0e0e8214d11867cc9a47c997b40`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:53.819655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8da50a1c-8435-4c42-8f42-c88bf1e557fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:53.915688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 944/1883] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:53.929578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:53.932785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:53.999762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a46f4c-a91b-45fa-bfa7-acc75c508724 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:54.003269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08f902-9ea6-4d5a-a9a0-ac37fc399445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:54.003907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08f902-9ea6-4d5a-a9a0-ac37fc399445 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:54.004181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08f902-9ea6-4d5a-a9a0-ac37fc399445 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:54.004496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08f902-9ea6-4d5a-a9a0-ac37fc399445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:54.016261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08f902-9ea6-4d5a-a9a0-ac37fc399445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:54.018055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08f902-9ea6-4d5a-a9a0-ac37fc399445 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:54.029655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 940/1884] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:43:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:43:54.458302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968bc9-8f9a-498f-8a17-c8805f64b938 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:54.458718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968bc9-8f9a-498f-8a17-c8805f64b938 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:54.459066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968bc9-8f9a-498f-8a17-c8805f64b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:54.477785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-03968bc9-8f9a-498f-8a17-c8805f64b938 None None] MFA Rules not processed for user `2e63c3f83f3d41018e6330f26ed7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:54.479542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-03968bc9-8f9a-498f-8a17-c8805f64b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:54.532826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 945/1885] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:43:57.570487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:57.571464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:57.572060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:57.593067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] MFA Rules not processed for user `cccdf0e0e8214d11867cc9a47c997b40`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:57.594628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:57.609209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 941/1886] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:43:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:57.609716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9cd79-1371-43e7-a177-195e4415ebeb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:57.610288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9cd79-1371-43e7-a177-195e4415ebeb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:57.610288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9cd79-1371-43e7-a177-195e4415ebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:57.633195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-36b9cd79-1371-43e7-a177-195e4415ebeb None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:43:57.634576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-36b9cd79-1371-43e7-a177-195e4415ebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:57.694321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 946/1887] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:43:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:43:57.737788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:43:57.739435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:43:57.823913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:43:57.841900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288874d6-8072-4b75-bec2-ac7fdd6f49a2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:43:57.847770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:43:57.848526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:43:57.848916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:43:57.849191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:43:57.858503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:43:57.860143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:43:57.875847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 942/1888] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:43:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:44:07.688178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=88 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:07.865495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:16.359994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60a7f-8c7c-4f23-805c-e073d0f2c876 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:16.360658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60a7f-8c7c-4f23-805c-e073d0f2c876 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:16.360658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60a7f-8c7c-4f23-805c-e073d0f2c876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:16.395452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3cc60a7f-8c7c-4f23-805c-e073d0f2c876 None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:16.396728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3cc60a7f-8c7c-4f23-805c-e073d0f2c876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:16.415284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 947/1889] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:44:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:16.420792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:16.436646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b518776-efd4-42a3-86a8-318e945eafd0 service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:16.454457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:16.456933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:16.457390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:16.457834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:16.480831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:16.482677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:16.493510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 943/1890] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:44:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:44:23.232967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558881-57ea-4781-a9bc-625faf47a11d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:23.233549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558881-57ea-4781-a9bc-625faf47a11d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:23.233549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558881-57ea-4781-a9bc-625faf47a11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:23.266390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e4558881-57ea-4781-a9bc-625faf47a11d None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:23.267676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e4558881-57ea-4781-a9bc-625faf47a11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:23.279466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 948/1891] 10.222.0.39 () {58 vars in 967 bytes} [Tue Oct 11 06:44:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:23.293105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:23.314724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fd911-56ae-4556-a32f-b0b014a4c78c service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:23.321656 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d610af5-5a20-469f-acbb-8e5e917b6c0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:23.322542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d610af5-5a20-469f-acbb-8e5e917b6c0a service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:23.322825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d610af5-5a20-469f-acbb-8e5e917b6c0a service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:23.323098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d610af5-5a20-469f-acbb-8e5e917b6c0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:23.332960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d610af5-5a20-469f-acbb-8e5e917b6c0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:23.334592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d610af5-5a20-469f-acbb-8e5e917b6c0a service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:23.347460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 944/1892] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:44:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:44:26.094655 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442eec-646c-4581-be8d-56cd70b21713 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:26.095790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442eec-646c-4581-be8d-56cd70b21713 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:26.096177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442eec-646c-4581-be8d-56cd70b21713 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:26.097712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 949/1893] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:44:26.154630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:26.155628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:26.155910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:26.182318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] MFA Rules not processed for user `8d6eb8eb03314f6aa7fe0ba171babf57`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:26.183650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:26.252033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442eec-646c-4581-be8d-56cd70b21713 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:26.253791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-25442eec-646c-4581-be8d-56cd70b21713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:26.281868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442eec-646c-4581-be8d-56cd70b21713 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:26.286847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:26.287736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:26.292666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:26.292666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] PATH_INFO: `/v3/users/0a48c619ff834ca7942d9be275fe61ea` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:26.294724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a48c619ff834ca7942d9be275fe61ea)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:26.298571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:26.303564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 945/1894] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:44:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:26.311034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:26.313327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:26.332474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cd692-f672-41dd-b47c-8d82de544e38 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:26.340451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:26.343214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:26.343961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:26.344541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:26.363871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:26.365920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:26.388739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a48c619ff834ca7942d9be275fe61ea'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:26.390422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a48c619ff834ca7942d9be275fe61ea'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:26.391638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a48c619ff834ca7942d9be275fe61ea'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:26.407620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 946/1895] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:26.440789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a48c619ff834ca7942d9be275fe61ea'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:26.452565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2520140a-0fe6-4c9e-9491-981419c3069d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a48c619ff834ca7942d9be275fe61ea'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:26.488113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 950/1896] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:44:26 2022] DELETE /identity/v3/users/0a48c619ff834ca7942d9be275fe61ea => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:27.159970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:27.180985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc413f5-9a18-4487-b4c3-69bd484cd03d service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:27.293531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:27.294216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:27.294761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:27.295269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] PATH_INFO: `/v3/projects/bc43c2d6c1364f23b21d203467adb5c5` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:27.296560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc43c2d6c1364f23b21d203467adb5c5)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:27.298802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:27.378775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc43c2d6c1364f23b21d203467adb5c5', 'request_id': 'req-429d9855-cf75-4d44-b9c3-75350b86185b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:27.386043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc43c2d6c1364f23b21d203467adb5c5', 'request_id': 'req-429d9855-cf75-4d44-b9c3-75350b86185b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:27.399593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:27.419993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc43c2d6c1364f23b21d203467adb5c5', 'request_id': 'req-429d9855-cf75-4d44-b9c3-75350b86185b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:27.440794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] The token cache is being invalidate because project bc43c2d6c1364f23b21d203467adb5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:44:27.441089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:27.441760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-429d9855-cf75-4d44-b9c3-75350b86185b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:27.457286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 947/1897] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:44:27 2022] DELETE /identity/v3/projects/bc43c2d6c1364f23b21d203467adb5c5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 06:44:27.957627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f475-68cb-498c-a48a-60f681bc7a16 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:27.958198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f475-68cb-498c-a48a-60f681bc7a16 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:27.958198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f475-68cb-498c-a48a-60f681bc7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:27.978196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f705f475-68cb-498c-a48a-60f681bc7a16 None None] MFA Rules not processed for user `3d57230baeb24e8a8a511a0c6417077e`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:27.979715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f705f475-68cb-498c-a48a-60f681bc7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:28.057842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 951/1898] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:35.267431 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce391aa-e5ef-47a8-b716-bf96a2ff98c0 None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.268381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce391aa-e5ef-47a8-b716-bf96a2ff98c0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.269115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce391aa-e5ef-47a8-b716-bf96a2ff98c0 None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.270819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 948/1899] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:44:35.281664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0ae4c-559a-42ea-9798-8bb192110b5b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.282198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0ae4c-559a-42ea-9798-8bb192110b5b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.283543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0ae4c-559a-42ea-9798-8bb192110b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.304922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8ec0ae4c-559a-42ea-9798-8bb192110b5b None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:35.307069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8ec0ae4c-559a-42ea-9798-8bb192110b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:35.368565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:35.392234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 952/1900] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:44:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:35.407608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce391aa-e5ef-47a8-b716-bf96a2ff98c0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:35.412025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3ce391aa-e5ef-47a8-b716-bf96a2ff98c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:35.483141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce391aa-e5ef-47a8-b716-bf96a2ff98c0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:35.486629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09249c1-bb32-4420-a1cd-7d260d147e5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:35.487229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09249c1-bb32-4420-a1cd-7d260d147e5d service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.487534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09249c1-bb32-4420-a1cd-7d260d147e5d service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.487819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09249c1-bb32-4420-a1cd-7d260d147e5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.498904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09249c1-bb32-4420-a1cd-7d260d147e5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:35.500391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09249c1-bb32-4420-a1cd-7d260d147e5d service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:35.511082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 949/1901] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:35.581582 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ddf8-cb67-4fe6-969c-22a36b7e586a None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.581937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ddf8-cb67-4fe6-969c-22a36b7e586a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.582210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ddf8-cb67-4fe6-969c-22a36b7e586a None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.584160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 953/1902] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:44:35.589040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5a06c-2e7c-4247-831b-6ec82cf32843 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.589326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5a06c-2e7c-4247-831b-6ec82cf32843 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.589639 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5a06c-2e7c-4247-831b-6ec82cf32843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.610741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cce5a06c-2e7c-4247-831b-6ec82cf32843 None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:35.611973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cce5a06c-2e7c-4247-831b-6ec82cf32843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:35.676015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 950/1903] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:44:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:44:35.692999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ddf8-cb67-4fe6-969c-22a36b7e586a None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:35.694796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d115ddf8-cb67-4fe6-969c-22a36b7e586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:35.711128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ddf8-cb67-4fe6-969c-22a36b7e586a None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:35.715130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a402a-6900-4c52-aa33-5ca2c5804fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:35.715872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a402a-6900-4c52-aa33-5ca2c5804fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.716149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a402a-6900-4c52-aa33-5ca2c5804fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.716469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a402a-6900-4c52-aa33-5ca2c5804fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.726825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a402a-6900-4c52-aa33-5ca2c5804fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:35.728568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a402a-6900-4c52-aa33-5ca2c5804fc8 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:35.737296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 954/1904] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:35.837397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f618d04-3dbf-465b-a73f-f7dfb1cbbe9f None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.837692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f618d04-3dbf-465b-a73f-f7dfb1cbbe9f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.837968 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f618d04-3dbf-465b-a73f-f7dfb1cbbe9f None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.839340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 951/1905] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 06:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 06:44:35.844225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.844510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.844784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.864602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:35.865804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:35.875045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 955/1906] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 06:44:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:35.885320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f618d04-3dbf-465b-a73f-f7dfb1cbbe9f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:35.896138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f618d04-3dbf-465b-a73f-f7dfb1cbbe9f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:35.900208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f910667-7868-4ac5-a189-485e5cec1313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:35.900832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f910667-7868-4ac5-a189-485e5cec1313 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:35.901147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f910667-7868-4ac5-a189-485e5cec1313 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:35.901485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f910667-7868-4ac5-a189-485e5cec1313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:35.912068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f910667-7868-4ac5-a189-485e5cec1313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:35.913605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f910667-7868-4ac5-a189-485e5cec1313 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:35.931119 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 952/1907] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:35.998927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:36.000513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:36.103378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a43fcc-9e1f-441e-a95b-80d31bfa5ec3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:36.115302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:36.116056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:36.116393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:36.116765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] PATH_INFO: `/v3/users/405122a8524e4ab78cc7c3c260f5ec5f` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:36.118046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405122a8524e4ab78cc7c3c260f5ec5f)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:36.120739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:36.177117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405122a8524e4ab78cc7c3c260f5ec5f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:36.177768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405122a8524e4ab78cc7c3c260f5ec5f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:36.178416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405122a8524e4ab78cc7c3c260f5ec5f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:36.187779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405122a8524e4ab78cc7c3c260f5ec5f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:36.198561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405122a8524e4ab78cc7c3c260f5ec5f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:36.221979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 956/1908] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:44:35 2022] DELETE /identity/v3/users/405122a8524e4ab78cc7c3c260f5ec5f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:36.227039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:36.227822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:36.227822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:36.249988 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:36.251274 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:36.331260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 953/1909] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:36.341571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:36.343055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:36.401813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356c1aa-585f-4caa-8847-be5d2e852ed8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:36.405826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c795-f6ef-488a-8bfb-5740b8f5115c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:36.406437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c795-f6ef-488a-8bfb-5740b8f5115c service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:36.406717 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c795-f6ef-488a-8bfb-5740b8f5115c service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:36.406994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c795-f6ef-488a-8bfb-5740b8f5115c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:36.416310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c795-f6ef-488a-8bfb-5740b8f5115c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:36.417742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c795-f6ef-488a-8bfb-5740b8f5115c service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:36.421743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:36.429483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 957/1910] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:36.907837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:36.909855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:36.926944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021472d7-6ed0-4eab-9571-86fa26d2089b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:36.933229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:36.933910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:36.934186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:36.934630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] PATH_INFO: `/v3/projects/d691da1e1a4045dc9255e1dd61d248b3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:36.936123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d691da1e1a4045dc9255e1dd61d248b3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:36.938051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:36.966404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:36.997330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd691da1e1a4045dc9255e1dd61d248b3', 'request_id': 'req-5be9689f-9076-4f18-af72-f815482a1ceb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.004231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd691da1e1a4045dc9255e1dd61d248b3', 'request_id': 'req-5be9689f-9076-4f18-af72-f815482a1ceb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.012903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd691da1e1a4045dc9255e1dd61d248b3', 'request_id': 'req-5be9689f-9076-4f18-af72-f815482a1ceb'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.025891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] The token cache is being invalidate because project d691da1e1a4045dc9255e1dd61d248b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:44:37.026247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.026985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5be9689f-9076-4f18-af72-f815482a1ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.032719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 954/1911] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:44:36 2022] DELETE /identity/v3/projects/d691da1e1a4045dc9255e1dd61d248b3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:37.033974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.034411 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.034855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.055480 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:37.057182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:37.127958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 958/1912] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:37.135942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.136260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.136593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.158330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:37.159666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:37.168126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 955/1913] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:37.172532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.183736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc863e5-b517-4264-9f47-d2a2ce6fa96c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.188282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.188887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.189160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.189428 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.190073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:44:37.190378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.192002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.209313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 959/1914] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:44:37.210529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.220089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be27c7-33cc-4a0f-82ed-51c793d8a83f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.224731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.225338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.225616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.226413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.227046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.228987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.258016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 956/1915] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:44:37 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:44:37.262166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.272604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135223ae-894c-4cd7-a922-554bc4b8a402 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.333278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.333943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.334158 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.334429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.335066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.337571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.362444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 960/1916] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:44:37 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 06:44:37.366576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.378011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1d538-aa5b-4466-89d6-2036f6437a24 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.383502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.384370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.384653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.384926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.385574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.387495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.401611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 957/1917] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:44:37.402906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.414906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed23d6-fa11-439f-906f-06ce63b0fa38 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.420098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac343b4-ee35-405c-8c78-d20a34f65606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.420624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac343b4-ee35-405c-8c78-d20a34f65606 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.420939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac343b4-ee35-405c-8c78-d20a34f65606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.421190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac343b4-ee35-405c-8c78-d20a34f65606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.421802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac343b4-ee35-405c-8c78-d20a34f65606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.423371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac343b4-ee35-405c-8c78-d20a34f65606 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.434552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 961/1918] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:44:37.438674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.450123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3868a69-3658-463b-b3cf-eab52a61fbf0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.455661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.456317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.456602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.456876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] PATH_INFO: `/v3/projects/2eab14d51b2b4afbae7333ffd8f8eee2/users/a2ecc040021448778dcd248539c85aa2/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.473600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eab14d51b2b4afbae7333ffd8f8eee2, user_id=a2ecc040021448778dcd248539c85aa2, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.476604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.490711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 958/1919] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:44:37 2022] PUT /identity/v3/projects/2eab14d51b2b4afbae7333ffd8f8eee2/users/a2ecc040021448778dcd248539c85aa2/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:37.501583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.501979 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.502282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.531350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] MFA Rules not processed for user `a2ecc040021448778dcd248539c85aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:37.532600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:37.590149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:37.606303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 962/1920] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:37.626903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.628693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:37.692244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d158-b7d0-43b9-aa6d-8e04e0aec8b9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.696151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fa0b9-6823-4f3b-880d-83f24fb8c480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.696762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fa0b9-6823-4f3b-880d-83f24fb8c480 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.697052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fa0b9-6823-4f3b-880d-83f24fb8c480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.697303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fa0b9-6823-4f3b-880d-83f24fb8c480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.706747 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fa0b9-6823-4f3b-880d-83f24fb8c480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.708208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fa0b9-6823-4f3b-880d-83f24fb8c480 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.717214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 959/1921] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:44:37.747475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:37.825470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b75ae-dda9-42d7-8e15-f7671f6a4f65 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:37.830439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:37.831093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.831397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.832098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] PATH_INFO: `/v3/users/a2ecc040021448778dcd248539c85aa2` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.833384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ecc040021448778dcd248539c85aa2)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:37.835787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:37.882574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ecc040021448778dcd248539c85aa2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.883163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ecc040021448778dcd248539c85aa2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.883797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ecc040021448778dcd248539c85aa2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.891120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ecc040021448778dcd248539c85aa2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.898327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ecc040021448778dcd248539c85aa2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:37.917579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 963/1922] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:44:37 2022] DELETE /identity/v3/users/a2ecc040021448778dcd248539c85aa2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:37.926412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119b453-204d-494c-b449-2536271b2db7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:37.926864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119b453-204d-494c-b449-2536271b2db7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:37.927200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119b453-204d-494c-b449-2536271b2db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:37.950850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c119b453-204d-494c-b449-2536271b2db7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:37.952102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c119b453-204d-494c-b449-2536271b2db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:37.998069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:38.023948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 960/1923] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:38.030462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.031858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:38.099350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ed52-3ac7-48b0-8e2a-b6711fcdf41c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.103581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2f187-8e5d-456d-a051-109c43e157a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.104180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f187-8e5d-456d-a051-109c43e157a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.104454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f187-8e5d-456d-a051-109c43e157a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.104723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f187-8e5d-456d-a051-109c43e157a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.113271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2f187-8e5d-456d-a051-109c43e157a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:38.114672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2f187-8e5d-456d-a051-109c43e157a4 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:38.123819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 964/1924] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:38.430409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119b453-204d-494c-b449-2536271b2db7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.432106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c119b453-204d-494c-b449-2536271b2db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:38.449929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119b453-204d-494c-b449-2536271b2db7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.456417 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.458032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.458121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.458518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] PATH_INFO: `/v3/projects/2eab14d51b2b4afbae7333ffd8f8eee2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.465181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eab14d51b2b4afbae7333ffd8f8eee2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:38.471294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:38.525751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eab14d51b2b4afbae7333ffd8f8eee2', 'request_id': 'req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:38.532934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eab14d51b2b4afbae7333ffd8f8eee2', 'request_id': 'req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:38.541978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eab14d51b2b4afbae7333ffd8f8eee2', 'request_id': 'req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:38.553610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] The token cache is being invalidate because project 2eab14d51b2b4afbae7333ffd8f8eee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:44:38.554087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:38.554922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cbb7ed0-6fac-43fd-bf6e-c3e7d451f9f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:44:38.556765 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 961/1925] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:44:38 2022] DELETE /identity/v3/projects/2eab14d51b2b4afbae7333ffd8f8eee2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:38.561562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.562100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.562100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.586335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:38.587686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:38.601873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:38.658083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 965/1926] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:38.668543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.668827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.669100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.695499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:38.696937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:38.705289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 962/1927] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:38.709484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.720736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf200a7-2ffd-4b62-a0f5-b436df0d2b2b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.726082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.726751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.726986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.727350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.728197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 06:44:38.728540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:38.730289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:38.744921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 966/1928] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 06:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 06:44:38.750313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.760387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676998ee-f9b5-4260-a326-a73150fb3604 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.764930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.765535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.765808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.766076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.768048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:38.771557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:38.806266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 963/1929] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 06:44:38 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:44:38.810731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.823271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c339a3-4a4d-4263-bf88-3a26c9cf581b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.902350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.903250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.903580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.903851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.904496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:38.906416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:38.929811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 967/1930] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 06:44:38 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 06:44:38.934110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.950269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c68b-44a2-465c-bc42-e6f99a0655b0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.956111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.956833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.957202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.957564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.958606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:38.960845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:38.976032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 964/1931] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:44:38.977853 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:38.992579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd279b-e533-4a14-bf0f-a1b19e43da5b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:38.997746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f02436-834c-4d26-b4ce-4cac4b91ae30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:38.998452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02436-834c-4d26-b4ce-4cac4b91ae30 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:38.998730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02436-834c-4d26-b4ce-4cac4b91ae30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:38.999002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f02436-834c-4d26-b4ce-4cac4b91ae30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:38.999741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f02436-834c-4d26-b4ce-4cac4b91ae30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:39.001404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f02436-834c-4d26-b4ce-4cac4b91ae30 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:39.015157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 968/1932] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 06:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 06:44:39.019018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:39.033730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688f112-d121-4347-b28f-b4e547cce211 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:39.043813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:39.044512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:39.044793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:39.045128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] PATH_INFO: `/v3/projects/8c0cbe1cb0974454a2c4c7cbde3681e8/users/66a6c4a5e3f94f1c932fd3210ae9bf05/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:39.062292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0cbe1cb0974454a2c4c7cbde3681e8, user_id=66a6c4a5e3f94f1c932fd3210ae9bf05, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:39.065035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:39.082473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 965/1933] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 06:44:39 2022] PUT /identity/v3/projects/8c0cbe1cb0974454a2c4c7cbde3681e8/users/66a6c4a5e3f94f1c932fd3210ae9bf05/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:44:39.099225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1cfff-b2f0-4427-9776-a70ffa755c4d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:39.099628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1cfff-b2f0-4427-9776-a70ffa755c4d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:39.099961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1cfff-b2f0-4427-9776-a70ffa755c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:39.138090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d0c1cfff-b2f0-4427-9776-a70ffa755c4d None None] MFA Rules not processed for user `66a6c4a5e3f94f1c932fd3210ae9bf05`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:39.139793 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d0c1cfff-b2f0-4427-9776-a70ffa755c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:39.219088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 969/1934] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:39.230160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:39.231984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:39.294353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e102-eba9-4188-a954-9450b3ed8297 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:39.298062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7613c-f03d-467e-af7e-2728ad60a318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:39.298669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7613c-f03d-467e-af7e-2728ad60a318 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:39.298944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7613c-f03d-467e-af7e-2728ad60a318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:39.299213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7613c-f03d-467e-af7e-2728ad60a318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:39.307957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7613c-f03d-467e-af7e-2728ad60a318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:39.309378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7613c-f03d-467e-af7e-2728ad60a318 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:39.318033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 966/1935] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:44:40.696160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:40.696726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:40.696726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:40.716055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] MFA Rules not processed for user `66a6c4a5e3f94f1c932fd3210ae9bf05`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:40.717257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:40.724537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 970/1936] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:44:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:40.873819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:40.875362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:40.875661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:40.895839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:44:40.896969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:40.905076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:40.978299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 967/1937] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:44:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:44:40.993982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:40.995801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:41.002247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:41.066507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e043-11ce-40f0-9fd4-0dcec644aad6 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:41.071438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fdcb4-0b5a-45b1-854a-6cc6d5721600 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:41.072076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fdcb4-0b5a-45b1-854a-6cc6d5721600 service swift] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:41.072352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fdcb4-0b5a-45b1-854a-6cc6d5721600 service swift] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:41.072621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fdcb4-0b5a-45b1-854a-6cc6d5721600 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:41.081040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fdcb4-0b5a-45b1-854a-6cc6d5721600 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:41.082607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fdcb4-0b5a-45b1-854a-6cc6d5721600 service swift] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:41.095010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 971/1938] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:44:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:44:50.969453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=20 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:51.090995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=30 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:44:55.767566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:44:55.769023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:55.849476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd621-dead-46e8-b46e-788aabaa224b None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:44:55.855209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:44:55.856087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:44:55.856224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:44:55.856401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:44:55.858125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:44:55.869557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:44:55.871209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:44:55.881493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 968/1939] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:44:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:45:05.877575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=32 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:45:15.243472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cbab-9c05-4060-8aac-e6046cd8718a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:15.243472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cbab-9c05-4060-8aac-e6046cd8718a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:15.244142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50cbab-9c05-4060-8aac-e6046cd8718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:15.265902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ae50cbab-9c05-4060-8aac-e6046cd8718a None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:45:15.267171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ae50cbab-9c05-4060-8aac-e6046cd8718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:45:15.274909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 972/1940] 10.222.0.39 () {58 vars in 967 bytes} [Tue Oct 11 06:45:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 06:45:15.297526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:45:15.310811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c0d7b-3a9f-4201-8698-3ab95e963718 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:45:15.316402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:45:15.317121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:15.317681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:15.318074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:15.328235 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:45:15.329966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:45:15.338802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 969/1941] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:45:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:45:21.650487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3fc7-7527-4397-9c5a-30118c3643dc None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:21.651017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3fc7-7527-4397-9c5a-30118c3643dc None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:21.651017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3fc7-7527-4397-9c5a-30118c3643dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:21.669856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d41b3fc7-7527-4397-9c5a-30118c3643dc None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:45:21.671023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d41b3fc7-7527-4397-9c5a-30118c3643dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:45:21.677932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 973/1942] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:45:21.689821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:45:21.700509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbce113-13e4-491a-9700-4c2d8f8e28cf service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:45:21.705482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:45:21.706064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:21.706337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:21.706602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:21.714943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:45:21.716530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:45:21.724468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 970/1943] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:45:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:45:22.234862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a1655-40c4-43d5-9c90-9fe8a300caec None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:22.235143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a1655-40c4-43d5-9c90-9fe8a300caec None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:22.235459 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a1655-40c4-43d5-9c90-9fe8a300caec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:22.254964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b83a1655-40c4-43d5-9c90-9fe8a300caec None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:45:22.256124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b83a1655-40c4-43d5-9c90-9fe8a300caec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:45:22.263076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 974/1944] 10.222.0.39 () {58 vars in 967 bytes} [Tue Oct 11 06:45:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:45:22.275086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:45:22.285522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb931350-84f4-45e6-86cf-6d14f3c6d310 service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:45:22.290955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:45:22.291574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:22.291851 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:22.292119 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:22.299831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:45:22.301692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:45:22.310300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 971/1945] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:45:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 11 06:45:22.529948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898fa7b-70d3-48a5-b89d-3421172f95a6 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:22.530232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898fa7b-70d3-48a5-b89d-3421172f95a6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:22.530500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898fa7b-70d3-48a5-b89d-3421172f95a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:22.549971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8898fa7b-70d3-48a5-b89d-3421172f95a6 None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:45:22.551168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8898fa7b-70d3-48a5-b89d-3421172f95a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:45:22.558857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 975/1946] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 06:45:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:45:22.570519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:45:22.580875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629273e3-0757-4abf-b997-d5c200f86e0a service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:45:22.586361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:45:22.586953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:22.587232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:22.587541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:22.595329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:45:22.596922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:45:22.605383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 972/1947] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:45:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:45:22.921700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:22.922226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:22.922226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:22.950804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:45:22.952148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:45:22.959664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 976/1948] 10.222.0.39 () {58 vars in 967 bytes} [Tue Oct 11 06:45:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:45:22.972922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:45:22.986195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537aca7-f12e-4445-8b0c-867e066e18ef service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:45:22.991846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:45:22.992439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:45:22.992716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:45:22.993186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:45:23.001189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:45:23.002752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:45:23.010887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 973/1949] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 06:45:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 06:45:32.946439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=15 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:45:33.007428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=30 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:46:49.620223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:46:49.621591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:49.633256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f856e-8edd-46c5-9f8e-06fb1d40b53d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:46:49.636739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:46:49.637328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:46:49.637601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:46:49.637869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:46:49.639521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:49.697908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:46:49.699331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:46:49.708203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 977/1950] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:46:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:46:52.021598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:46:52.023065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:52.094354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4988-3107-454a-ae27-eb6fc5487b95 service swift] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:46:52.099671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:46:52.100265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:46:52.100539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:46:52.100805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:46:52.103035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:52.159147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:46:52.160687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:46:52.169895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 974/1951] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:46:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:46:53.568291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:46:53.569604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:53.584515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02928693-8ff9-436f-a41f-fecbf997bccf service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:46:53.589724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:46:53.590308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:46:53.590586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:46:53.590854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:46:53.592490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:53.652998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:46:53.654583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:46:53.663676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 978/1952] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:46:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:46:56.923869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:46:56.933013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea2f2-44d6-431e-bfe1-dfa2b598eae7 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:46:56.936558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:46:56.937143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:46:56.937432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:46:56.937682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:46:56.939361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:46:56.997901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:46:56.999373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:46:57.008494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 975/1953] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:46:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:47:03.659296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=56 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:47:07.004180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=84 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:47:50.766231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:47:50.767657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:47:50.825391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a398a-6edd-4f94-b4ac-4358ff692015 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:47:50.828897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:47:50.829501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:47:50.829770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:47:50.830036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:47:50.831756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:47:50.840738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:47:50.842140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:47:50.849787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 979/1954] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:47:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:48:00.846379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=26 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:48:53.379478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:48:53.388542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb285cb-7b2d-4ee7-9cff-71e56fde6e42 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:48:53.392107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:48:53.392704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:48:53.393005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:48:53.393277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:48:53.394944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:48:53.455041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:48:53.456530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:48:53.465509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 976/1955] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:48:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:49:03.461411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=28 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:49:38.951507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:49:38.960644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ffce4-6eb3-494d-a038-3bfada5fd5ea service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:49:38.964726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:49:38.965320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:49:38.965598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:49:38.965866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:49:38.973918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:49:38.975324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:49:38.984590 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 980/1956] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:49:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:49:48.980057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:50:53.104142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:50:53.163388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1857-bf93-4bc1-8b3e-2e99dfbcb4d6 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:50:53.168392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:50:53.168992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:50:53.169268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:50:53.169537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:50:53.177971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:50:53.179645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:50:53.189280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 977/1957] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:50:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:03.185202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=27 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:51:15.302367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:15.329409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ba11-161d-4c95-9738-ffadcfdf4a8f service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:15.332963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:15.333552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:15.333830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:15.334100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:15.335826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:15.376770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:15.378174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d538f7-8852-45fd-9999-988edbd69c8d service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:15.397885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 981/1958] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:23.617054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:23.699568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952e3d4-3ff0-4dda-a500-64997b1b6349 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:23.704178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:23.704801 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:23.705078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:23.705348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] PATH_INFO: `/v3/users/66a6c4a5e3f94f1c932fd3210ae9bf05` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:23.706457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a6c4a5e3f94f1c932fd3210ae9bf05)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:23.708415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:23.755409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a6c4a5e3f94f1c932fd3210ae9bf05'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:23.756088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a6c4a5e3f94f1c932fd3210ae9bf05'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:23.763961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a6c4a5e3f94f1c932fd3210ae9bf05'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:23.764614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a6c4a5e3f94f1c932fd3210ae9bf05'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:23.772477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a6c4a5e3f94f1c932fd3210ae9bf05'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:23.789597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 978/1959] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 06:51:23 2022] DELETE /identity/v3/users/66a6c4a5e3f94f1c932fd3210ae9bf05 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:51:23.794594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:23.795085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:23.795269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:23.833701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 06:51:23.834885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:23.891934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 982/1960] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 06:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 06:51:23.901537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:23.902977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:23.970862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:51:23.981717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a17e9-ffd9-4532-b6c9-f594a2c55aad admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:23.985369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:23.985965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:23.986244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:23.986515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:23.996056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:23.997486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:24.006625 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 979/1961] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:24.181122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:24.182519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:24.195247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1778e1c-cee8-4411-b676-1fb4a72b398f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:24.200026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:24.200689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:24.201085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:24.201367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] PATH_INFO: `/v3/projects/8c0cbe1cb0974454a2c4c7cbde3681e8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:24.202600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0cbe1cb0974454a2c4c7cbde3681e8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:24.204574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:24.252944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0cbe1cb0974454a2c4c7cbde3681e8', 'request_id': 'req-9ad2710c-6705-4fee-a68c-b31463f19df4'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:24.259867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0cbe1cb0974454a2c4c7cbde3681e8', 'request_id': 'req-9ad2710c-6705-4fee-a68c-b31463f19df4'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:24.267132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0cbe1cb0974454a2c4c7cbde3681e8', 'request_id': 'req-9ad2710c-6705-4fee-a68c-b31463f19df4'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:24.274065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:51:24.280196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] The token cache is being invalidate because project 8c0cbe1cb0974454a2c4c7cbde3681e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 06:51:24.280683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:24.281839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 06:51:24.283553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 983/1962] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 06:51:24 2022] DELETE /identity/v3/projects/8c0cbe1cb0974454a2c4c7cbde3681e8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 06:51:34.002522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:51:34.284564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:51:48.019168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:48.028194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b46bb-c371-4b1c-a288-02b2ed5ca743 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:48.077064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:48.077706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:48.078000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:48.078260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:48.080045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:48.149511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:48.151456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:48.162210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 980/1963] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:49.998420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:49.999861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:50.015875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2710c-6705-4fee-a68c-b31463f19df4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:50.021258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:50.021855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:50.022140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:50.022414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:50.024152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:50.104177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:50.105760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:50.123678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 984/1964] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:52.604487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:52.605866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:52.674020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4081f-18d2-4822-b593-77cefe6fcbb3 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:52.677539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:52.678137 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:52.678412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:52.678707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:52.680428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:52.727520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:51:52.763790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:52.765473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:52.785016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 981/1965] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:53.336282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:53.346195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad00e-cbad-4efa-b3ef-89802f902724 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:53.349736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:53.350328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:53.350603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:53.350871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:53.352595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:53.430448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:53.432415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:53.441160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 985/1966] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:51:57.485731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:51:57.498383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3dd4c2-1661-477d-8858-df7845e86456 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:51:57.503827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:51:57.504420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:51:57.504694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:51:57.504966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:51:57.506711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:51:57.589229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:51:57.590821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:51:57.608832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 982/1967] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:51:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:52:03.437029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=78 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:52:07.606340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=61 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:52:53.806491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:52:53.808145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:52:53.820002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3552939-af62-428f-b196-9f3de0e89532 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:52:53.823739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:52:53.824367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:52:53.824644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:52:53.824913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:52:53.826626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:52:53.836453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:52:53.838000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:52:53.846078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 986/1968] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:52:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 06:53:03.842510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:53:54.471699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:53:54.473197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:53:54.484634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6a087-b5d6-4412-9148-b14d7a197e4c service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:53:54.488163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:53:54.488781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:53:54.489057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:53:54.489325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:53:54.490974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:53:54.568885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:53:54.570309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:53:54.589277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 983/1969] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:53:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:54:04.586046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=41 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:55:58.119102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:55:58.120733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:55:58.219046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d992ba-73c2-4dff-991f-c03c81d9c63a service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:55:58.224103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:55:58.224708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:55:58.224985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:55:58.225255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:55:58.226968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:55:58.238636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:55:58.240245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:55:58.249896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 987/1970] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:55:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:56:08.246011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=37 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:56:48.490770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:56:48.500070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdee08-8bb5-4ce1-b763-352ec3be549b service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:56:48.503682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:56:48.504279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:56:48.504557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:56:48.504865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:56:48.515186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:56:48.516657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:56:48.527760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 984/1971] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:56:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:56:50.501258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:56:50.502653 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:56:50.517811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe16f09-a051-402f-b32f-05f0f87dbe36 service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:56:50.523208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:56:50.523827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:56:50.524108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:56:50.524380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:56:50.532542 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:56:50.534108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:56:50.543538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 988/1972] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:56:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:56:53.173538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:56:53.182631 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9de4f-057d-47dc-8e30-cec136a0962c service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:56:53.187303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:56:53.187512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:56:53.187712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:56:53.187981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:56:53.196487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:56:53.197914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:56:53.207164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 985/1973] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:56:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:56:53.834488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:56:53.844348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a004df-9dcd-4ec6-b745-3c5f8fc70823 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:56:53.847891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:56:53.848486 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:56:53.848764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:56:53.849034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:56:53.857198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:56:53.858607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:56:53.877208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 989/1974] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:56:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:56:57.838950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:56:57.851039 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc19dd-716a-4efd-a1be-f2b143c085a0 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:56:57.856418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:56:57.856998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:56:57.857270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:56:57.857535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:56:57.865111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:56:57.866698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:56:57.876096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 986/1975] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:56:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:57:03.874619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=16 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:57:07.871304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=18 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:57:16.661681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:57:16.663003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:57:16.674557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac46e2-55ae-4a72-b77c-7302548466d8 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:57:16.678077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:57:16.678669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:57:16.678945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:57:16.679212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:57:16.680885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:57:16.693966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:57:16.695368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:57:16.706500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 990/1976] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:57:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:57:26.701989 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:57:47.033891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:57:47.046592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534a416-d3d9-444b-8375-2bedfc4c2fae service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:57:47.052198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:57:47.052799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:57:47.053193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:57:47.053473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:57:47.055191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 06:57:47.068874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:57:47.070458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:57:47.082298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 987/1977] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:57:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:57:57.078245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:58:54.780346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:58:54.788942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad327d7-974c-4919-9024-30ffbc895954 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:58:54.792587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:58:54.793185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:58:54.793465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:58:54.793734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:58:54.802618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:58:54.804079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:58:54.813711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 991/1978] 10.222.0.39 () {60 vars in 1396 bytes} [Tue Oct 11 06:58:54 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:59:03.805895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 06:59:03.815754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e188baf-1370-43c0-8934-23737d35caee service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 06:59:03.819220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 06:59:03.819867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 06:59:03.820193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 06:59:03.820465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 06:59:03.827439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 06:59:03.828927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 06:59:03.837146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 988/1979] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 06:59:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 06:59:04.808930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 06:59:13.833725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:00:58.260849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:00:58.262266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:00:58.320439 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08daea-f4d6-478d-a4f7-788870cbb06c service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:00:58.326557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:00:58.327366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:00:58.327813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:00:58.328152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:00:58.337266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:00:58.338828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:00:58.358408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 992/1980] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:00:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:01:08.353194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=23 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:01:48.449230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:01:48.521164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fec6e6-3507-4f42-9f4d-9b0b39b48c5a service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:01:48.525280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:01:48.525898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:01:48.526158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:01:48.526424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:01:48.528304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:01:48.636401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:01:48.638109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:01:48.658739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 989/1981] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:01:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:01:51.252550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:01:51.283281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790b186-713d-491c-90d4-87685f29b4bd service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:01:51.286802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:01:51.287400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:01:51.287712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:01:51.287984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:01:51.290025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:01:51.368606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:01:51.370019 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:01:51.388303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 993/1982] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:01:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:01:53.145802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:01:53.147363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:01:53.160891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e77702-ed30-4fc0-96ab-37c6d36c7541 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:01:53.166941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:01:53.167598 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:01:53.167878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:01:53.168165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:01:53.169946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:01:53.215891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:01:53.249503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:01:53.251095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:01:53.269533 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 990/1983] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:01:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:01:53.480100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:01:53.489208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3684-2dce-4988-aea7-a1de50f139d6 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:01:53.493423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:01:53.494015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:01:53.494291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:01:53.494559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:01:53.496409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:01:53.583733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:01:53.585684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:01:53.594941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 994/1984] 10.222.0.39 () {60 vars in 1396 bytes} [Tue Oct 11 07:01:53 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:01:57.806267 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:01:57.860214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ddb71-6e76-43a0-baa7-479902d98ba2 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:01:57.863781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:01:57.864381 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:01:57.864661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:01:57.864934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:01:57.867359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:01:57.946975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:01:57.948492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:01:57.966135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 991/1985] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:01:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:02:03.590622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=87 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:02:07.963588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=81 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:02:16.817056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:02:16.825618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196c20-4c08-469d-87d3-eec92494cc7c service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:02:16.829292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48854d77-0dff-499e-b6bb-f242d5af7344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:02:16.829891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48854d77-0dff-499e-b6bb-f242d5af7344 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:02:16.830169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48854d77-0dff-499e-b6bb-f242d5af7344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:02:16.830440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48854d77-0dff-499e-b6bb-f242d5af7344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:02:16.841398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48854d77-0dff-499e-b6bb-f242d5af7344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:02:16.842811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48854d77-0dff-499e-b6bb-f242d5af7344 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:02:16.854309 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 995/1986] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:02:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:02:26.848851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:03:16.027331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:03:16.049017 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e43be2-b7f9-4f61-89e4-6ddef804ae1d service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:03:16.057475 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:03:16.058374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:03:16.058810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:03:16.059260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:03:16.071613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:03:16.073312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:03:16.086084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 992/1987] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:03:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:03:24.554017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:03:24.554519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:03:24.554639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:03:24.582835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] MFA Rules not processed for user `2e63c3f83f3d41018e6330f26ed7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:03:24.584068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:03:24.591883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 996/1988] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:03:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:03:24.893477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:03:24.895095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:03:24.918837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a966-6220-411b-817e-754d000d78d2 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:03:24.922388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:03:24.922986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:03:24.923261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:03:24.923566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:03:24.925261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:03:24.935029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:03:24.936479 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:03:24.944471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 993/1989] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:03:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:03:26.271867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:03:26.284265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2d2a4-b226-4c45-91aa-8d2e743710d0 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:03:26.289189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:03:26.289791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:03:26.290071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:03:26.290349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:03:26.292070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:03:26.303644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:03:26.305209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721949-d709-4cdf-b8b5-6b1c02ee539a service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:03:26.315134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 997/1990] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:03:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:03:34.940877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=17 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:03:36.310731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=12 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:03:37.952520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:03:37.967467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406ee87-f392-43d6-a533-51cb327fb781 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:03:37.972877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:03:37.973481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:03:37.973759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:03:37.974028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:03:37.975789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:03:37.987995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:03:37.989846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:03:38.001328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 994/1991] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:03:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:03:47.996735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:03:53.112074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:03:53.112603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:03:53.112715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:03:53.141711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] MFA Rules not processed for user `f95c1cd09e1042abbb5725f241ce147d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:03:53.142911 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:03:53.150244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 998/1992] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:03:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:04:03.139436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:04:05.612627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:04:05.622045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaabc1-34d6-4deb-a920-8e8131d303ca service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:04:05.625618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:04:05.626256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:04:05.626531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:04:05.626802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:04:05.628697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:04:05.638834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:04:05.640294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:04:05.648261 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 995/1993] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:04:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:04:15.644725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:05:16.133807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:05:16.135613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:05:16.159735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf95e65-7e09-415c-a7ca-a9e73e7a0976 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:05:16.163217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:05:16.163843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:05:16.164120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:05:16.164388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:05:16.166141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:05:16.244465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:05:16.245872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:05:16.264449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 999/1994] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:05:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:05:26.261899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=46 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:06:06.118077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:06:06.183744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63b906-7b5f-4d1f-98a3-12748eda8fe9 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:06:06.189275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:06:06.189874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:06:06.190150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:06:06.190419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:06:06.192262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:06:06.215222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:06:06.216889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:06:06.226786 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 996/1995] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:06:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:06:16.222602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=37 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:06:49.899198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:06:49.908248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1fc77-4852-47a7-be00-1f23f98d7e0c service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:06:49.911756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:06:49.912345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:06:49.912616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:06:49.912881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:06:49.923156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:06:49.924611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:06:49.935201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1000/1996] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:06:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:06:51.000181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:06:51.001830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:06:51.016792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d11d2-1197-480b-87a1-16f9958451e1 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:06:51.022144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:06:51.022739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:06:51.023013 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:06:51.023283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:06:51.030945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:06:51.032547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:06:51.041249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 997/1997] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:06:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:06:59.930521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:07:01.037217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:07:16.809781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:07:16.819879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f77f3-ca9b-4d92-a940-1c0a80c24226 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:07:16.823441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:07:16.824034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:07:16.824310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:07:16.824578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:07:16.832779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:07:16.834188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:07:16.852028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1001/1998] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:07:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:07:16.894840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:07:16.903844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68643552-01ad-49b7-97e8-e6406633255d service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:07:16.907342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:07:16.907954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:07:16.908252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:07:16.908499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:07:16.917138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:07:16.918545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:07:16.927889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 998/1999] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:07:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:07:16.988603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:07:16.996822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeedd67-11b7-4c8c-aa0d-086c60fc68b9 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:07:17.000441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:07:17.001028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:07:17.001304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:07:17.001571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:07:17.010009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:07:17.011408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:07:17.020375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1002/2000] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:07:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:07:17.561301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:07:17.562921 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:07:17.574898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c9df1-c9d2-48b3-8650-3d7232c53131 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:07:17.578788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:07:17.579383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:07:17.579687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:07:17.579952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:07:17.582278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:07:17.595176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:07:17.597400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94637d3e-b732-4290-a0c4-ec516061211c service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:07:17.608454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 999/2001] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:07:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:07:27.016133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=16 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:07:27.603698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=12 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:09:12.814257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:09:12.824297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ee10-755d-4021-a020-8600cca5bbb7 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:09:12.827886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:09:12.828479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:09:12.828757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:09:12.829026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:09:12.836032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:09:12.837455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:09:12.845183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1003/2002] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:09:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:09:16.881767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:09:16.882314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:09:16.882314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:09:16.910827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] MFA Rules not processed for user `1c602e0f9e0e4e479a6a3c32febdb14b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:09:16.912069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:09:16.918884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1000/2003] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:09:17.226839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:09:17.235366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13f0a-c480-4303-bd3c-808763b43caf service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:09:17.239046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:09:17.239661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:09:17.239942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:09:17.240740 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:09:17.248041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:09:17.249460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:09:17.257975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1004/2004] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:09:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:09:17.574347 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:09:17.586401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd413d-af4a-480b-9ea1-44087eb3e3d7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:09:17.591315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:09:17.591986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:09:17.592265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:09:17.592537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:09:17.602445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:09:17.604114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:09:17.614389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1001/2005] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:09:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:09:18.538257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:09:18.551879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc2249-acde-4e77-867b-6448e66780c5 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:09:18.562152 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:09:18.562864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:09:18.563218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:09:18.563601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:09:18.565857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:09:18.581096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:09:18.582807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:09:18.594147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1005/2006] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:09:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:09:25.384958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:09:25.397983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d42a83-2322-4225-91e8-b83e5170424e service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:09:25.405557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:09:25.405557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:09:25.405557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:09:25.405557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:09:25.417292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:09:25.418888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:09:25.430480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1002/2007] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:09:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:09:28.589302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=18 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:09:35.425186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=18 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:05.954804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:05.957078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:06.018921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9dee-008f-465a-a1dd-f9c36a658f55 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:06.024182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:06.025330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:06.025610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:06.025883 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:06.034397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:06.036026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:06.046201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1006/2008] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:11:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:11:16.041663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=23 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:18.526779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:18.528992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:18.541327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc48a3a-91f9-4047-881a-3372fc278b67 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:18.545152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:18.545755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:18.546033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:18.546297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:18.554814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:18.556346 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:18.565520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1003/2009] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:11:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:11:21.024598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:21.038096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09240663-5048-4685-8c15-3631940c2a5b service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:21.052897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:21.054350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:21.054698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:21.055212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:21.058217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:21.072957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:21.074763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be17eea-e82d-4ae0-baf8-c17022c3ff30 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:21.087132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1007/2010] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:11:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:11:24.647712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:24.659597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd7745-a14e-4cfb-a21e-9b256f37bbf6 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:24.665317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:24.666002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:24.666341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:24.666795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:24.668580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:24.767002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:24.769088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:24.795712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1004/2011] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:11:26.378226 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:26.378742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:26.378742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:26.398201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] MFA Rules not processed for user `2e63c3f83f3d41018e6330f26ed7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:11:26.399994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:26.408282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1008/2012] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:11:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:11:26.804575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:26.814877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242a0eb-317b-45cd-bb3a-c0f5b12d2f67 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:26.818908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:26.821009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:26.821318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:26.821722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:26.823712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:26.835896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:26.837838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:26.847373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1005/2013] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:11:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:11:28.301916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:28.319891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c5ebf-a2d9-49b6-92a1-0e7f94f1be86 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:28.325802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:28.326782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:28.327068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:28.327345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:28.329690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:28.349125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:28.352403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:28.369341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:28.371242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:28.386216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1009/2014] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:11:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:11:28.389880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3119e-5f3a-4c9e-b142-b7648dfed9a6 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:28.394704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:28.395334 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:28.395691 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:28.395964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] PATH_INFO: `/v3/users/3d57230baeb24e8a8a511a0c6417077e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:28.417038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d57230baeb24e8a8a511a0c6417077e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:28.424056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:28.492639 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d57230baeb24e8a8a511a0c6417077e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.493657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d57230baeb24e8a8a511a0c6417077e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.502596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d57230baeb24e8a8a511a0c6417077e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.503288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d57230baeb24e8a8a511a0c6417077e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.511400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-51cc3194-3f65-4d59-a9b0-c7aaa8e7c4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d57230baeb24e8a8a511a0c6417077e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.529797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1006/2015] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:11:28 2022] DELETE /identity/v3/users/3d57230baeb24e8a8a511a0c6417077e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:11:28.708247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:28.728644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf553540-e90f-4ccf-bc41-f7e42b735998 service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:28.789471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:28.790117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:28.790396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:28.790673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] PATH_INFO: `/v3/projects/0e4e977dc2e84b08a4fbaf7fd2fac42f` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:28.791897 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4e977dc2e84b08a4fbaf7fd2fac42f)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:28.794179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:28.845684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4e977dc2e84b08a4fbaf7fd2fac42f', 'request_id': 'req-41283d63-ddfc-4820-a0a2-95a970ba6dc8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.854006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4e977dc2e84b08a4fbaf7fd2fac42f', 'request_id': 'req-41283d63-ddfc-4820-a0a2-95a970ba6dc8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.861764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4e977dc2e84b08a4fbaf7fd2fac42f', 'request_id': 'req-41283d63-ddfc-4820-a0a2-95a970ba6dc8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.873183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] The token cache is being invalidate because project 0e4e977dc2e84b08a4fbaf7fd2fac42f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:11:28.873472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.874172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-41283d63-ddfc-4820-a0a2-95a970ba6dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:11:28.876018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1010/2016] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:11:28 2022] DELETE /identity/v3/projects/0e4e977dc2e84b08a4fbaf7fd2fac42f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:11:28.882412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:28.882774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:28.883708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:28.900051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:28.946972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:11:28.948251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:29.007780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1007/2017] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:11:29.016255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.016537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.016812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.036144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:11:29.037362 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:29.045196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1011/2018] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:11:29.049878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.060642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cef8a-ef18-4020-aac9-771f1517cfae None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.065312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.065909 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.066183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.066453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.067095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:11:29.067396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.069076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.085373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1008/2019] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 07:11:29.086505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.095696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7d351-ee52-4510-99b4-b3481a297023 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.100326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.100920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.101193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.101460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.102096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.104027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.146150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1012/2020] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:11:29.147302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.157642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524b52-28f7-4ae7-a914-6c2345396005 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.217705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.218291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.218572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.218843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.219518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.221512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.245507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1009/2021] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:11:29.249689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.260509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e1cc4-68de-4c49-b88d-cd3fdd9fc52c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.265449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.266075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.266360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.266628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.267275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.268912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.279903 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1013/2022] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:11:29.284123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.296761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdb8a4-c733-4514-91f7-f9857c6c7461 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.301816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ecc55-bdd3-4ea6-ac74-8193b8ead9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.302437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecc55-bdd3-4ea6-ac74-8193b8ead9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.302741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecc55-bdd3-4ea6-ac74-8193b8ead9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.303014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecc55-bdd3-4ea6-ac74-8193b8ead9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.303701 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecc55-bdd3-4ea6-ac74-8193b8ead9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.305944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecc55-bdd3-4ea6-ac74-8193b8ead9a6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.317430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1010/2023] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:11:29.321646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.331833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0190e21-940c-44a0-beaf-5a88472a6c3d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.336527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.337241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.337518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.337786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] PATH_INFO: `/v3/projects/0877a2806ea54e799c45401285a4e8ae/users/5f93a8618fc441cab6640098cbcbd2ab/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.341726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:29.356952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0877a2806ea54e799c45401285a4e8ae, user_id=5f93a8618fc441cab6640098cbcbd2ab, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.359675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.374892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1014/2024] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:11:29 2022] PUT /identity/v3/projects/0877a2806ea54e799c45401285a4e8ae/users/5f93a8618fc441cab6640098cbcbd2ab/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:11:29.386400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.386926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.386926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.419209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] MFA Rules not processed for user `5f93a8618fc441cab6640098cbcbd2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:11:29.420452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:29.458548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:29.500505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1011/2025] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:11:29.505545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.574547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826bfd-aad4-4608-b840-9835a5c9c149 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.579613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.580469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.580749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.581022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.581691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.583649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.620423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1015/2026] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:11:29.624037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.636036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c33e8-0016-4427-a9cc-38f6ab30c2fd None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.708053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.708743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.709085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.709424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.710708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.713554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.748866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1012/2027] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:11:29.751086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.765537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046a37e-6028-4ecd-b1a4-2d82fade83d0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.770382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.771085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.771484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.771861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.772584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.774988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.787023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1016/2028] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:11:29.791717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.801833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53f63-2b91-4622-a9e2-a8c60cfc0462 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.806644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.807292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.807602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.807876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] PATH_INFO: `/v3/projects/fc20dd63ce734dd0b1accc9d7f0d0cb0/users/25c6b7205e794a0e8411590c6daa8d22/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.822814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc20dd63ce734dd0b1accc9d7f0d0cb0, user_id=25c6b7205e794a0e8411590c6daa8d22, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.825539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.840302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1013/2029] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:11:29 2022] PUT /identity/v3/projects/fc20dd63ce734dd0b1accc9d7f0d0cb0/users/25c6b7205e794a0e8411590c6daa8d22/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:11:29.844198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.854912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1dc55-3684-4290-8af4-f018aab6efff admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.922001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aba5a4-98a5-4dbd-a1b8-6403aa33f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.922620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba5a4-98a5-4dbd-a1b8-6403aa33f48c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.922900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba5a4-98a5-4dbd-a1b8-6403aa33f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.923173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba5a4-98a5-4dbd-a1b8-6403aa33f48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.923855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aba5a4-98a5-4dbd-a1b8-6403aa33f48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.925441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aba5a4-98a5-4dbd-a1b8-6403aa33f48c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.935312 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1017/2030] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:11:29.939490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:11:29.950727 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29623-fb5d-462b-a63e-adf07778342f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:11:29.955946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:11:29.958726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.958726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.958981 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] PATH_INFO: `/v3/projects/fc20dd63ce734dd0b1accc9d7f0d0cb0/users/25c6b7205e794a0e8411590c6daa8d22/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:29.961012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc20dd63ce734dd0b1accc9d7f0d0cb0, user_id=25c6b7205e794a0e8411590c6daa8d22, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:11:29.964344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:11:29.975774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1014/2031] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:11:29 2022] PUT /identity/v3/projects/fc20dd63ce734dd0b1accc9d7f0d0cb0/users/25c6b7205e794a0e8411590c6daa8d22/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:11:29.985769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41f94f-10da-486e-aa8e-5af8732a5b60 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:11:29.985981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41f94f-10da-486e-aa8e-5af8732a5b60 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:11:29.986252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41f94f-10da-486e-aa8e-5af8732a5b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:11:30.014517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-be41f94f-10da-486e-aa8e-5af8732a5b60 None None] MFA Rules not processed for user `25c6b7205e794a0e8411590c6daa8d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:11:30.016080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-be41f94f-10da-486e-aa8e-5af8732a5b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:11:30.059890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:30.087642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1018/2032] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:11:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:11:39.971720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:11:40.083919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=9 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:30.159146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:30.222516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e9af7-0ef1-40e8-9dd2-0005fa7ad464 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:30.227453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:30.228064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:30.228445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:30.228712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] PATH_INFO: `/v3/users/5f93a8618fc441cab6640098cbcbd2ab` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:30.229807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f93a8618fc441cab6640098cbcbd2ab)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:30.231928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:30.279679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f93a8618fc441cab6640098cbcbd2ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.280326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f93a8618fc441cab6640098cbcbd2ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.287064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f93a8618fc441cab6640098cbcbd2ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.287709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f93a8618fc441cab6640098cbcbd2ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.295636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f93a8618fc441cab6640098cbcbd2ab'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.312507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1015/2033] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:12:30 2022] DELETE /identity/v3/users/5f93a8618fc441cab6640098cbcbd2ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:12:30.316963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:30.317467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:30.317576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:30.345850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:12:30.347050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:30.404465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1019/2034] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:12:30.414217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:30.415586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:30.486840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454564d6-4232-439a-80fd-720e3daedc9d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:30.490462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:30.491057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:30.491331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:30.492131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:30.501187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:30.502591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:30.512252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1016/2035] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:12:30.778864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:30.780307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:30.792395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c23cd2-bd5c-4a26-8ef6-0a3a09e18194 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:30.797453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:30.798175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:30.798529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:30.798818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] PATH_INFO: `/v3/projects/0877a2806ea54e799c45401285a4e8ae` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:30.800112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=0877a2806ea54e799c45401285a4e8ae)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:30.802089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:30.849373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0877a2806ea54e799c45401285a4e8ae', 'request_id': 'req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.857016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0877a2806ea54e799c45401285a4e8ae', 'request_id': 'req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.863377 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0877a2806ea54e799c45401285a4e8ae', 'request_id': 'req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.873671 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] The token cache is being invalidate because project 0877a2806ea54e799c45401285a4e8ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:12:30.873985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.874978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.876572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1020/2036] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:12:30 2022] DELETE /identity/v3/projects/0877a2806ea54e799c45401285a4e8ae => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:12:30.880629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:30.885892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:30.892172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113e770-67a5-45bc-ac86-585f5beb1605 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:30.948571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:30.949123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:30.949403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:30.949676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] PATH_INFO: `/v3/users/25c6b7205e794a0e8411590c6daa8d22` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:30.950781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c6b7205e794a0e8411590c6daa8d22)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:30.952728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:30.998240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c6b7205e794a0e8411590c6daa8d22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:30.998874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c6b7205e794a0e8411590c6daa8d22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.005827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c6b7205e794a0e8411590c6daa8d22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.006445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c6b7205e794a0e8411590c6daa8d22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.014219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-41551f93-f919-4c17-a631-48f73876af2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c6b7205e794a0e8411590c6daa8d22'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.031182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1017/2037] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:12:30 2022] DELETE /identity/v3/users/25c6b7205e794a0e8411590c6daa8d22 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:12:31.293166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.295045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:31.308465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62149e-40d1-42fa-9b80-c0e32cbaa0be admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.365077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.365760 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.365975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.366246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] PATH_INFO: `/v3/projects/fc20dd63ce734dd0b1accc9d7f0d0cb0` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.368007 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc20dd63ce734dd0b1accc9d7f0d0cb0)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.369936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.386270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:31.420437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc20dd63ce734dd0b1accc9d7f0d0cb0', 'request_id': 'req-d58bc406-b887-4994-96c1-35aee351b79c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.427709 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc20dd63ce734dd0b1accc9d7f0d0cb0', 'request_id': 'req-d58bc406-b887-4994-96c1-35aee351b79c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.434672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc20dd63ce734dd0b1accc9d7f0d0cb0', 'request_id': 'req-d58bc406-b887-4994-96c1-35aee351b79c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.445056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] The token cache is being invalidate because project fc20dd63ce734dd0b1accc9d7f0d0cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:12:31.445344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.445971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d58bc406-b887-4994-96c1-35aee351b79c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:12:31.447473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1021/2038] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:12:31 2022] DELETE /identity/v3/projects/fc20dd63ce734dd0b1accc9d7f0d0cb0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:12:31.452420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.452906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.453020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.472248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:12:31.473383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:31.531728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1018/2039] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:12:31.541353 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.541637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.541910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.561920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:12:31.563127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:31.571001 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1022/2040] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:12:31.575121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.584870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf7c3b-de2b-49d2-b230-ad2ec76e0cc9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.589464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.590057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.590336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.590611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.591262 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:12:31.591606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.593214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.605116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1019/2041] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:12:31.609209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.618966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ec0fa-9537-4a6c-9f9b-119bc9fcecd7 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.623642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.624492 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.624773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.625048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.625677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.627639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.653996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1023/2042] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:12:31 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:12:31.657934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.668203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b06b83-515f-4070-8df2-d208c18832a3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.672951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:31.726683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.727206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.727512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.727779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.728418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.730370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.751880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1020/2043] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:12:31 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:12:31.755931 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.765871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb299-529d-4d97-b690-233929fe83d8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.770389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.770987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.771262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.771569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.772204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.774296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.784061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1024/2044] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:12:31.787918 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.798541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a85bf-396f-4a88-98f8-4fb98dfa8390 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.803320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b630e3-1a78-448f-9345-d1b62f44cdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.803953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b630e3-1a78-448f-9345-d1b62f44cdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.804233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b630e3-1a78-448f-9345-d1b62f44cdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.804505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b630e3-1a78-448f-9345-d1b62f44cdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.805150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b630e3-1a78-448f-9345-d1b62f44cdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.806733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b630e3-1a78-448f-9345-d1b62f44cdd7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.817351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1021/2045] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:12:31.821291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:31.830953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace00e1-0c08-4373-84b4-c3121ffc1433 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:31.836048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:31.836678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.836947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.837214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] PATH_INFO: `/v3/projects/7774c3dcab2f4f73825f39c483d98e3a/users/615fa09750b14e3aac64e44ec2636101/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.851645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7774c3dcab2f4f73825f39c483d98e3a, user_id=615fa09750b14e3aac64e44ec2636101, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:31.854273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:31.866922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1025/2046] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:12:31 2022] PUT /identity/v3/projects/7774c3dcab2f4f73825f39c483d98e3a/users/615fa09750b14e3aac64e44ec2636101/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:12:31.877241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:31.877530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:31.877807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:31.905755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] MFA Rules not processed for user `615fa09750b14e3aac64e44ec2636101`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:12:31.906929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:31.968756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1022/2047] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:12:31.973142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:32.033547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8f79-809a-4fdf-bb71-1a8b0d7f05d6 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:32.038073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:32.039271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.039577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.039848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.040499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:32.042387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:32.067267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1026/2048] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:12:31 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:12:32.071290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:32.081841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309862e9-1863-4057-8e06-710e4f73df3c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:32.138129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:32.138731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.139007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.139272 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.139931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:32.141849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:32.162601 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1023/2049] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:12:32 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:12:32.166554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:32.176518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff730e-d868-4180-90c8-8d0ddc551e27 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:32.181049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:32.181639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.181943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.182216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.182850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:32.184460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:32.196347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1027/2050] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:12:32.200358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:32.210161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84194370-b228-4a41-830c-63e6a6fd8050 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:32.215364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:32.216049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.216322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.216586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] PATH_INFO: `/v3/projects/7615d636377f4c45ad6345e29348b429/users/35730311bb2140c38969ba690bf2d48d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.218739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:32.232622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7615d636377f4c45ad6345e29348b429, user_id=35730311bb2140c38969ba690bf2d48d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:32.235296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:32.247664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1024/2051] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:12:32 2022] PUT /identity/v3/projects/7615d636377f4c45ad6345e29348b429/users/35730311bb2140c38969ba690bf2d48d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:12:32.251806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:32.262068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be4b07-840f-4d38-acd5-84b1c603d306 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:32.274022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:32.319258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ebb9d-5d3f-467d-addd-48d643c2dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:32.320327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ebb9d-5d3f-467d-addd-48d643c2dbaa admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.320606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ebb9d-5d3f-467d-addd-48d643c2dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.320876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ebb9d-5d3f-467d-addd-48d643c2dbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.321513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ebb9d-5d3f-467d-addd-48d643c2dbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:32.323088 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ebb9d-5d3f-467d-addd-48d643c2dbaa admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:32.333072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1028/2052] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:12:32.337766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:12:32.347958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64645e4-01fd-4b4a-ad22-00a1af16edef admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:12:32.354416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:12:32.355056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.355341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.355643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] PATH_INFO: `/v3/projects/7615d636377f4c45ad6345e29348b429/users/35730311bb2140c38969ba690bf2d48d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.357552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7615d636377f4c45ad6345e29348b429, user_id=35730311bb2140c38969ba690bf2d48d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:12:32.360239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:12:32.370323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1025/2053] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:12:32 2022] PUT /identity/v3/projects/7615d636377f4c45ad6345e29348b429/users/35730311bb2140c38969ba690bf2d48d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:12:32.380499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802af15-4de3-488b-aa91-29766a480412 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:12:32.380981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802af15-4de3-488b-aa91-29766a480412 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:12:32.381183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802af15-4de3-488b-aa91-29766a480412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:12:32.410145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0802af15-4de3-488b-aa91-29766a480412 None None] MFA Rules not processed for user `35730311bb2140c38969ba690bf2d48d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:12:32.411343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0802af15-4de3-488b-aa91-29766a480412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:12:32.477607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1029/2054] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:12:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:12:42.366656 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=12,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:12:42.474742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=56 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:13:08.568959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:08.571351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:08.671874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1132459-e3d9-46c7-84e8-dea13803014b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:08.677197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:08.677795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:08.678071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:08.678341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:08.680556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:08.758265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:13:08.759878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:13:08.768642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1026/2055] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:13:08.945105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802af15-4de3-488b-aa91-29766a480412 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:08.946425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0802af15-4de3-488b-aa91-29766a480412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:09.005211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802af15-4de3-488b-aa91-29766a480412 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:09.008767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:09.009363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:09.009642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:09.009912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:09.011624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:09.024391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:13:09.025807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:13:09.036338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1030/2056] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:13:10.212774 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:10.214148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:10.225322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014bf7d-76eb-453a-8bcf-d905c647e770 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:10.228818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:10.229400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:10.229674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:10.229939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:10.231588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:10.244636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:13:10.246035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:13:10.257693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1027/2057] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:13:19.031682 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=26 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:13:19.231508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:19.233299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:19.302555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02fd1f-76fc-47aa-a089-6818e48ae565 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:19.306075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:19.306727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:19.307004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:19.307277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:19.309041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:19.390199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:13:19.391661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:13:19.410689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1031/2058] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:13:19.458025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:19.468150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b983d-ab42-4ef5-9f39-b266dafffc69 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:19.471927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:19.472520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:19.472796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:19.473068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:19.474759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:19.510382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:13:19.561445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:13:19.562821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:13:19.583176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1028/2059] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:13:19.624715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:19.633708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461b65e-fe29-4a0b-a534-db744d01611a service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:19.637318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:19.637917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:19.638197 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:19.638471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:19.640306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] Could not find project: fc20dd63ce734dd0b1accc9d7f0d0cb0.: keystone.exception.ProjectNotFound: Could not find project: fc20dd63ce734dd0b1accc9d7f0d0cb0. Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 11 07:13:19.653396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 07:13:19.654174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 07:13:19.655125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application with Lock( Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._enter() Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.creator() Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application created_value = creator( Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 07:13:19.656371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 07:13:19.657418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: fc20dd63ce734dd0b1accc9d7f0d0cb0. Oct 11 07:13:19.657418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: ERROR keystone.server.flask.application Oct 11 07:13:19.657418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1032/2060] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:19 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 11 07:13:20.182688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:13:20.191954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390ac2-2f86-4762-b0bf-d1df3d705750 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:13:20.196641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:13:20.197303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:13:20.197579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:13:20.197928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:13:20.207842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:13:20.209220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:13:20.219004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1029/2061] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:13:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:13:29.643796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=69 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:13:30.214663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=30 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:12.053355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:12.055581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:12.151745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c616ee-4f2b-447e-b9b1-84ac47b0756e service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:12.157078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:12.157689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:12.158000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:12.158235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:12.160076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:12.181810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:12.183543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:12.197522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1033/2062] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:12.302072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:12.303578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:12.326535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2dab-d276-411c-8a50-82a151f3db44 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:12.330041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:12.330629 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:12.330903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:12.331172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:12.389640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:12.391233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:12.400893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1030/2063] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:14.348381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:14.349785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:14.365856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222ad0a-a17a-42e2-be2e-7177f3f9ddb1 service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:14.371944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:14.372561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:14.372839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:14.373110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:14.374848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:14.391856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:14.394035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e0a31-381e-47fd-8350-0ee406518814 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:14.408523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1034/2064] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:15.077679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:15.143805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772ce52-3d5f-4962-af8a-d96c5b3d6f9e service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:15.148238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:15.148865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:15.149142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:15.149413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] PATH_INFO: `/v3/users/615fa09750b14e3aac64e44ec2636101` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:15.150527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] RBAC: Authorizing `identity:delete_user(user_id=615fa09750b14e3aac64e44ec2636101)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:15.152493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:15.199702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615fa09750b14e3aac64e44ec2636101'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.200385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615fa09750b14e3aac64e44ec2636101'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.208258 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615fa09750b14e3aac64e44ec2636101'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.208907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615fa09750b14e3aac64e44ec2636101'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.216028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615fa09750b14e3aac64e44ec2636101'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.233282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1031/2065] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:15 2022] DELETE /identity/v3/users/615fa09750b14e3aac64e44ec2636101 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:15.237825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:15.238711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:15.238986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:15.257779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:15.258971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:15.316436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1035/2066] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:15.326307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:15.329122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:15.340588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:15.404551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bd2a1-438d-4b1e-acba-d7a0fe8d324b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:15.408294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:15.408897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:15.409172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:15.409441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:15.421407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:15.423802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:15.434438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1032/2067] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:15.735735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:15.737270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:15.749912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eefb90-1084-42bc-860c-86136a869df4 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:15.754799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:15.755455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:15.755736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:15.756008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] PATH_INFO: `/v3/projects/7774c3dcab2f4f73825f39c483d98e3a` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:15.757124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7774c3dcab2f4f73825f39c483d98e3a)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:15.759025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:15.806367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7774c3dcab2f4f73825f39c483d98e3a', 'request_id': 'req-c830430f-482f-4e54-96d8-b4bb04d64eaa'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.813605 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7774c3dcab2f4f73825f39c483d98e3a', 'request_id': 'req-c830430f-482f-4e54-96d8-b4bb04d64eaa'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.821185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7774c3dcab2f4f73825f39c483d98e3a', 'request_id': 'req-c830430f-482f-4e54-96d8-b4bb04d64eaa'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.822470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:15.833801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] The token cache is being invalidate because project 7774c3dcab2f4f73825f39c483d98e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:15.834082 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.834761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.836307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1036/2068] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:15 2022] DELETE /identity/v3/projects/7774c3dcab2f4f73825f39c483d98e3a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:15.840538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:15.850478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df54d8c-5b8b-470b-b3f0-7d0aa00e736d service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:15.911160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:15.911756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:15.912033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:15.912401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] PATH_INFO: `/v3/users/35730311bb2140c38969ba690bf2d48d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:15.913575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35730311bb2140c38969ba690bf2d48d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:15.915820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:15.961140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35730311bb2140c38969ba690bf2d48d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.961771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35730311bb2140c38969ba690bf2d48d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.968561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35730311bb2140c38969ba690bf2d48d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.969176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35730311bb2140c38969ba690bf2d48d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.975973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-cf67f67b-bbe8-4789-98f2-1e117a73a785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35730311bb2140c38969ba690bf2d48d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:15.992610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1033/2069] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:15 2022] DELETE /identity/v3/users/35730311bb2140c38969ba690bf2d48d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:16.257341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.258811 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:16.272879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830430f-482f-4e54-96d8-b4bb04d64eaa admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.328552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.329175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.329447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.329715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] PATH_INFO: `/v3/projects/7615d636377f4c45ad6345e29348b429` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.330816 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7615d636377f4c45ad6345e29348b429)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.332803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.380875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7615d636377f4c45ad6345e29348b429', 'request_id': 'req-00f6cf6f-61f2-4883-94b0-101038b028a8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:16.388570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7615d636377f4c45ad6345e29348b429', 'request_id': 'req-00f6cf6f-61f2-4883-94b0-101038b028a8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:16.396646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7615d636377f4c45ad6345e29348b429', 'request_id': 'req-00f6cf6f-61f2-4883-94b0-101038b028a8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:16.426384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] The token cache is being invalidate because project 7615d636377f4c45ad6345e29348b429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:16.426669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:16.427341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-00f6cf6f-61f2-4883-94b0-101038b028a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:16.428839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1037/2070] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:16 2022] DELETE /identity/v3/projects/7615d636377f4c45ad6345e29348b429 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:16.433655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.434087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.434174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.453951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:16.455144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:16.467030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:16.524331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1034/2071] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:16.533262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.533667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.534072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.556912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:16.558108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:16.566475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1038/2072] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:16.570832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.580929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fa9bc-2a65-4f79-8bd8-fdd1ed5ac23a None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.585522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.586124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.586404 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.586675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.587354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:16:16.587783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.589377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.601446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1035/2073] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:16:16.605534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.615380 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d9f9-91d7-4a37-afdb-ca7179d9773d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.620205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.620790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.621063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.621331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.621953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.623868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.650705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1039/2074] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:16 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:16.654816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.665270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cebdab-660b-4483-bdeb-8511ccffc92d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.723233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.723910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.724191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.724460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.725100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.727024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.745985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.747767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:16.750143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1036/2075] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:16 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:16:16.753975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.763304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284be31d-9b1c-4bf8-96c1-58ef5ab3225e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.767897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.768473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.768752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.769020 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.769654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.771234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.781864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1037/2076] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:16.786136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.795541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf045-1bbb-4e2f-904f-a97cd751f13b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.800286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.801565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.801845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.802187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.802827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.804757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.804893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7145b0e-ba4f-4055-abfe-8a9310688005 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.808466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.809068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.809347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.809633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.811472 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:16.814996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1038/2077] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:16:16.818965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:16.820587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:16.825461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.827629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc2ebc-999a-4f6f-954c-36cce5504d52 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.828738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24600f34-c85b-484e-9b0f-e385fe9cbf28 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:16.835607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:16.836348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.836698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.837257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] PATH_INFO: `/v3/projects/93fcfc882cb44b98833edd33f735be2e/users/9bf8e79fa099497fa206ef26d281d938/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.839892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1040/2078] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:16.853005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=93fcfc882cb44b98833edd33f735be2e, user_id=9bf8e79fa099497fa206ef26d281d938, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:16.855784 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:16.869244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1039/2079] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:16 2022] PUT /identity/v3/projects/93fcfc882cb44b98833edd33f735be2e/users/9bf8e79fa099497fa206ef26d281d938/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:16.879551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:16.879825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:16.880094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:16.908261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] MFA Rules not processed for user `9bf8e79fa099497fa206ef26d281d938`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:16.909496 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:16.978329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1041/2080] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:16.982768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.033870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:17.049644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92eca3-033f-4c28-ac63-6e04628107ee admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.054226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.054826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.055108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.055369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.056031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.057926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.088153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1040/2081] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:16 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:17.092204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.104323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aeea5-ee1a-4ba2-86f1-40ed06003e40 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.162830 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.163479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.163764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.164040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.164718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.166640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.188835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1042/2082] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:17 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:17.193378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.203717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75002b-1f57-4a57-b236-a18caf38c1a8 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.208536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.209131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.209425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.209685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.210504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.212152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.223579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1041/2083] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:17.227588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.238955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be99d24-a338-4ef5-8875-9a4554d253fd admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.244738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.245579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.245878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.246238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] PATH_INFO: `/v3/projects/b2b8ade7ad9d4da1af83af22dd7b198c/users/76dd1ce28c4343109fbb2d579e07ad37/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.264308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b8ade7ad9d4da1af83af22dd7b198c, user_id=76dd1ce28c4343109fbb2d579e07ad37, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.267471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.283164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1043/2084] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:17 2022] PUT /identity/v3/projects/b2b8ade7ad9d4da1af83af22dd7b198c/users/76dd1ce28c4343109fbb2d579e07ad37/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:17.287796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.297852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d116286-948d-4cd5-8f42-6b84851c6250 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.306489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.308965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.367446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.367446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.367772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:17.367849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.367849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.367849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.368295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.379783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1042/2085] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:17.380478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.382746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.391817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4228b-fce3-42fb-a9dd-aa7c9e6e54c9 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.395051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f047ed-dbe8-44a3-9ff4-d5ab57174628 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.396909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.397608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.397890 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.398157 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.399661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.399912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.400276 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.400556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.400827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] PATH_INFO: `/v3/users/cccdf0e0e8214d11867cc9a47c997b40` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.412551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.414155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.418653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccdf0e0e8214d11867cc9a47c997b40)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.420641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.426563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1044/2086] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:17.427364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.439609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f74286-04c0-4a88-94f2-d9f7fbae0e28 service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.446351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06484d6f-d3e2-40c9-abbe-f9db8c74e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.447209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06484d6f-d3e2-40c9-abbe-f9db8c74e631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.447984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06484d6f-d3e2-40c9-abbe-f9db8c74e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.448796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06484d6f-d3e2-40c9-abbe-f9db8c74e631 admin admin] PATH_INFO: `/v3/projects/b2b8ade7ad9d4da1af83af22dd7b198c/users/76dd1ce28c4343109fbb2d579e07ad37/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.452210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06484d6f-d3e2-40c9-abbe-f9db8c74e631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b8ade7ad9d4da1af83af22dd7b198c, user_id=76dd1ce28c4343109fbb2d579e07ad37, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.455186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06484d6f-d3e2-40c9-abbe-f9db8c74e631 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.469329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1045/2087] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:17 2022] PUT /identity/v3/projects/b2b8ade7ad9d4da1af83af22dd7b198c/users/76dd1ce28c4343109fbb2d579e07ad37/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:17.488555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.488555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.488555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.509766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccdf0e0e8214d11867cc9a47c997b40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.510810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccdf0e0e8214d11867cc9a47c997b40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.514611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] MFA Rules not processed for user `76dd1ce28c4343109fbb2d579e07ad37`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:17.515844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.518440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccdf0e0e8214d11867cc9a47c997b40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.519105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccdf0e0e8214d11867cc9a47c997b40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.527923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccdf0e0e8214d11867cc9a47c997b40'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.546154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1043/2088] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:17 2022] DELETE /identity/v3/users/cccdf0e0e8214d11867cc9a47c997b40 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:17.590154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1046/2089] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:17.603432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.605098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.634742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:17.681715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54a32c-b771-4b9e-91b8-b1d7ca3812a5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.685852 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.686520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.686804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.687078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.740131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.741521 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.766316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.768122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.778456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1044/2090] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:17.811204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f860e0a-d6e2-4627-9cf2-254023dc4074 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.864124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:17.869844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:17.871378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.872227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.872507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.872778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] PATH_INFO: `/v3/projects/9ca42ed577ae40258f333875e0efbd62` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.881133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077cf98-857c-493f-84d5-b5f7e450118f service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:17.884470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca42ed577ae40258f333875e0efbd62)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.884821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:17.885415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.885687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.885977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.886394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.887743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:17.933888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca42ed577ae40258f333875e0efbd62', 'request_id': 'req-39473118-6b4a-429c-bc39-fddac4316b4f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.942576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca42ed577ae40258f333875e0efbd62', 'request_id': 'req-39473118-6b4a-429c-bc39-fddac4316b4f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.949487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca42ed577ae40258f333875e0efbd62', 'request_id': 'req-39473118-6b4a-429c-bc39-fddac4316b4f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.951770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:17.953229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc381f-34c5-4438-be1a-4264209446dc service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:17.963883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1045/2091] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:17.966441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] The token cache is being invalidate because project 9ca42ed577ae40258f333875e0efbd62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:17.966733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.967374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-39473118-6b4a-429c-bc39-fddac4316b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:17.968867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1047/2092] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:17 2022] DELETE /identity/v3/projects/9ca42ed577ae40258f333875e0efbd62 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:17.975372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:17.975688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:17.975966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:17.997096 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:17.998290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.060803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1046/2093] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:18.070232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.070517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.070789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.090967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:18.092208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.100876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1048/2094] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:18.105143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.115599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b2676-7583-4baa-a887-7131dbf7af8c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.121059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.121664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.121942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.122216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.122868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:16:18.123170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.124848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.127503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:18.140584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1047/2095] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:16:18.141766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.151399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef5420-eb7e-4c96-bb61-56cf917b0e7a None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.156037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.156666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.156942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.157409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.158042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.160565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.188960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1049/2096] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:18 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:18.193004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.203097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd277cbd-c428-4e49-8953-7c57bd062284 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.237040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.239343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.266623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.267157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.267455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.267735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.268358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.270280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.296428 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1048/2097] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:18 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:18.297693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.308212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d21da-2c06-46f2-acb7-2db8b6fae5f0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.308690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefebad3-5518-44d8-9bc6-2c5b69bd9698 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.312928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.313348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.313521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.313787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.313999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.314090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.314277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.314548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] PATH_INFO: `/v3/users/9bf8e79fa099497fa206ef26d281d938` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.314707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.315755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf8e79fa099497fa206ef26d281d938)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.316301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.317691 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.327234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1049/2098] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:16:18.330993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.340426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c1fcf-e8fb-4565-95ea-e9029aab3e54 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.345217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.345767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.346048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.346325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.346940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.349138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.359429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1050/2099] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:16:18.364069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.365659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf8e79fa099497fa206ef26d281d938'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:18.366333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf8e79fa099497fa206ef26d281d938'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:18.366955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf8e79fa099497fa206ef26d281d938'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:18.372178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:18.373832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54098f5a-0496-42be-99e5-162f49307803 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.376777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf8e79fa099497fa206ef26d281d938'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:18.378591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcb93-7e60-4142-ab91-3c1cecb75809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.379223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcb93-7e60-4142-ab91-3c1cecb75809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.379524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcb93-7e60-4142-ab91-3c1cecb75809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.379802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcb93-7e60-4142-ab91-3c1cecb75809 admin admin] PATH_INFO: `/v3/projects/60dd1d4ab8434282b5f57d6547f8d3e9/users/52173fbead6e4b499a99a6979c9a9898/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.385111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44e1b9fa-aef1-4780-8e0d-76e8bf6a6999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf8e79fa099497fa206ef26d281d938'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:18.395446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dcb93-7e60-4142-ab91-3c1cecb75809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60dd1d4ab8434282b5f57d6547f8d3e9, user_id=52173fbead6e4b499a99a6979c9a9898, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.398107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dcb93-7e60-4142-ab91-3c1cecb75809 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.403370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1050/2100] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:18 2022] DELETE /identity/v3/users/9bf8e79fa099497fa206ef26d281d938 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:18.407751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.408608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.408608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.417147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1051/2101] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:18 2022] PUT /identity/v3/projects/60dd1d4ab8434282b5f57d6547f8d3e9/users/52173fbead6e4b499a99a6979c9a9898/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:18.422200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.422478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.422879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.428102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:18.429338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.440999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:18.442289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.487194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1051/2102] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:18.497382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.498944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.505810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1052/2103] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:18.515089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.516967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.563215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ff5d7-2acc-49b1-a4c5-d8375c0f21ed None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.566923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.567559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.567835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.568105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.577239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.577894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:18.578745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.589299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1052/2104] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:18.596565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cbaa9-0b35-4467-9a73-e8c72a079d23 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.600799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.601408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.601686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.601956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.611317 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.612851 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:18.623056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1053/2105] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:18.935904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:18.937976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:18.958994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3713-df70-440d-999f-d02409ac1cf6 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:18.965399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:18.966112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:18.966460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:18.966851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] PATH_INFO: `/v3/projects/93fcfc882cb44b98833edd33f735be2e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:18.968182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93fcfc882cb44b98833edd33f735be2e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:18.970100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:19.030258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93fcfc882cb44b98833edd33f735be2e', 'request_id': 'req-f92ec106-93fc-4f2e-b6bf-071934641bc0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.041966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93fcfc882cb44b98833edd33f735be2e', 'request_id': 'req-f92ec106-93fc-4f2e-b6bf-071934641bc0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.049307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93fcfc882cb44b98833edd33f735be2e', 'request_id': 'req-f92ec106-93fc-4f2e-b6bf-071934641bc0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.055609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:19.063280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] The token cache is being invalidate because project 93fcfc882cb44b98833edd33f735be2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:19.063945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.065221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.068168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1053/2106] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:18 2022] DELETE /identity/v3/projects/93fcfc882cb44b98833edd33f735be2e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:19.071866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:19.082301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c46874-62b5-4da9-be2e-058641719741 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:19.145107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:19.145925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:19.146065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:19.146495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] PATH_INFO: `/v3/users/76dd1ce28c4343109fbb2d579e07ad37` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:19.147916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=76dd1ce28c4343109fbb2d579e07ad37)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:19.150196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:19.206972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76dd1ce28c4343109fbb2d579e07ad37'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.207745 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76dd1ce28c4343109fbb2d579e07ad37'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.215517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76dd1ce28c4343109fbb2d579e07ad37'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.216248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76dd1ce28c4343109fbb2d579e07ad37'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.224675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3c488ac1-87d7-4e3f-b921-ea164e3036ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76dd1ce28c4343109fbb2d579e07ad37'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.247973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1054/2107] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:19 2022] DELETE /identity/v3/users/76dd1ce28c4343109fbb2d579e07ad37 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:19.753782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:19.755485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:19.783909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec106-93fc-4f2e-b6bf-071934641bc0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:19.853676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:19.854358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:19.854637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:19.854915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] PATH_INFO: `/v3/projects/b2b8ade7ad9d4da1af83af22dd7b198c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:19.856108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b8ade7ad9d4da1af83af22dd7b198c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:19.858049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:19.919796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b8ade7ad9d4da1af83af22dd7b198c', 'request_id': 'req-0c496959-4e93-4c24-a514-55fc11a692ee'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.929801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b8ade7ad9d4da1af83af22dd7b198c', 'request_id': 'req-0c496959-4e93-4c24-a514-55fc11a692ee'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.938687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b8ade7ad9d4da1af83af22dd7b198c', 'request_id': 'req-0c496959-4e93-4c24-a514-55fc11a692ee'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.953184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] The token cache is being invalidate because project b2b8ade7ad9d4da1af83af22dd7b198c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:19.953527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.954712 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0c496959-4e93-4c24-a514-55fc11a692ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:19.957767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1054/2108] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:19 2022] DELETE /identity/v3/projects/b2b8ade7ad9d4da1af83af22dd7b198c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:19.962464 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd3d99-6103-4352-bada-611076020043 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:19.963380 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd3d99-6103-4352-bada-611076020043 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:19.963764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd3d99-6103-4352-bada-611076020043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:19.985596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c6bd3d99-6103-4352-bada-611076020043 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:19.986965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c6bd3d99-6103-4352-bada-611076020043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:20.054547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1055/2109] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:20.062731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.063022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.063298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.085200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:20.086344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:20.096736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1055/2110] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:16:20.104594 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd3d99-6103-4352-bada-611076020043 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.111587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:20.122225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd3d99-6103-4352-bada-611076020043 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.128537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.129210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.129486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.129754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.130463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:16:20.130768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.132686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.154237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1056/2111] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:16:20.158482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.169727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54258a9e-63d9-49cb-9fcf-e38908a441e3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.174512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.175124 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.175487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.175759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.176401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.178510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.215589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1056/2112] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:20 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:20.216275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.227374 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d42e-8d28-4c9b-8e4c-8ddf7b08283c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.291769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.292384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.292653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.293080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.294023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.295621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.321695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1057/2113] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:20 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:16:20.323836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.337899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b1a0-e2ff-45ea-9b4b-80e281a50b3e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.355771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.356402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.356677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.356945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.357599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.359244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.371531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1057/2114] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:20.375845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.390075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646a9a6-6fe3-4610-9d90-c6b9a3268e7c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.399632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13f5c4-3b37-4426-b727-87f454dc3365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.400270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13f5c4-3b37-4426-b727-87f454dc3365 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.403858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13f5c4-3b37-4426-b727-87f454dc3365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.404184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13f5c4-3b37-4426-b727-87f454dc3365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.404890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13f5c4-3b37-4426-b727-87f454dc3365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.406548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13f5c4-3b37-4426-b727-87f454dc3365 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.431180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1058/2115] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:20.435029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.445833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb87b4-b17c-4689-b610-3a318bdfe7ab admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.450987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.451695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.451976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.452248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] PATH_INFO: `/v3/projects/8ac83bdb2fee4d8caa4a6c7ccbba05aa/users/9f97a682031645f1a03ac65e1dcf1d59/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.481680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac83bdb2fee4d8caa4a6c7ccbba05aa, user_id=9f97a682031645f1a03ac65e1dcf1d59, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.485029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.504072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1058/2116] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:20 2022] PUT /identity/v3/projects/8ac83bdb2fee4d8caa4a6c7ccbba05aa/users/9f97a682031645f1a03ac65e1dcf1d59/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:20.514610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea034bd-d21e-4fa9-ae03-8d17240bc057 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.515180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea034bd-d21e-4fa9-ae03-8d17240bc057 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.515543 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea034bd-d21e-4fa9-ae03-8d17240bc057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.546511 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-dea034bd-d21e-4fa9-ae03-8d17240bc057 None None] MFA Rules not processed for user `9f97a682031645f1a03ac65e1dcf1d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:20.547744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dea034bd-d21e-4fa9-ae03-8d17240bc057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:20.615153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1059/2117] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:20.627183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:20.628995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:20.672114 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:20.743526 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3fac1-8693-4a57-9ad4-3800a18e2166 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:20.747537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:20.748150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:20.748916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:20.749184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:20.760419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:20.762329 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:20.773128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1059/2118] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:23.262864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.263446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.263682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.297207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] MFA Rules not processed for user `52173fbead6e4b499a99a6979c9a9898`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:23.299076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:23.347466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:23.376124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1060/2119] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:16:23.380488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:23.382042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:23.461894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ea13d-8bf7-4bca-88ab-b19511c9cafc service glance] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:23.466639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:23.467277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.467596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.467865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.468504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:23.470487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:23.500474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1060/2120] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:23 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:23.505116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:23.516580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6129c3-ff49-45cf-a07b-44a05a3f6968 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:23.577942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:23.578949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.579275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.579583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.580454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:23.583227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:23.607193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1061/2121] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:23 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:23.611273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:23.623834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b7f97-3d00-4d76-89ef-819351431665 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:23.629810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:23.630523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.630804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.631103 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.631839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:23.633456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:23.645301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1061/2122] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:23.649563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:23.660217 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf79008-426d-4198-8568-f69aa0fadc8d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:23.664779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:23.665420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.665696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.665983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] PATH_INFO: `/v3/projects/ee49cf21898f4f5894d61abbb9996acc/users/1261192c1b724bc8a0d020c5df609800/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.684221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee49cf21898f4f5894d61abbb9996acc, user_id=1261192c1b724bc8a0d020c5df609800, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:23.686902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:23.700995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1062/2123] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:23 2022] PUT /identity/v3/projects/ee49cf21898f4f5894d61abbb9996acc/users/1261192c1b724bc8a0d020c5df609800/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:23.705054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:23.716555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9fe21-3976-43a6-b6c2-35ba0c668438 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:23.785023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3ded2-dd2f-4b4d-b7d2-f0c327146914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:23.785929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3ded2-dd2f-4b4d-b7d2-f0c327146914 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.786210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3ded2-dd2f-4b4d-b7d2-f0c327146914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.786522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3ded2-dd2f-4b4d-b7d2-f0c327146914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.787179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3ded2-dd2f-4b4d-b7d2-f0c327146914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:23.788824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3ded2-dd2f-4b4d-b7d2-f0c327146914 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:23.798681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1062/2124] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:23.803458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:23.813777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70dfa8-2330-49e3-be10-d1b125887c07 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:23.819531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:23.820226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:23.820502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:23.820797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] PATH_INFO: `/v3/projects/ee49cf21898f4f5894d61abbb9996acc/users/1261192c1b724bc8a0d020c5df609800/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:23.822704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee49cf21898f4f5894d61abbb9996acc, user_id=1261192c1b724bc8a0d020c5df609800, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:23.825414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:23.836425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1063/2125] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:23 2022] PUT /identity/v3/projects/ee49cf21898f4f5894d61abbb9996acc/users/1261192c1b724bc8a0d020c5df609800/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:28.785572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:28.786110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:28.786317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:28.820870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] MFA Rules not processed for user `1261192c1b724bc8a0d020c5df609800`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:28.822714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:28.827898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:28.909330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1063/2126] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:28.926972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:28.928828 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:29.010652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10130dc-4895-48d0-b3d1-d502ee4048e2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:29.016270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae3fc-204f-4a02-a350-d65006ad770d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:29.016884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae3fc-204f-4a02-a350-d65006ad770d service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:29.017165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae3fc-204f-4a02-a350-d65006ad770d service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:29.017437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae3fc-204f-4a02-a350-d65006ad770d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:29.057922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:29.085501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae3fc-204f-4a02-a350-d65006ad770d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:29.087123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae3fc-204f-4a02-a350-d65006ad770d service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:29.096818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1064/2127] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:38.905583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=27 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:39.092308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=11 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:41.807727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:41.869902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146de7a-edb7-4abb-afe6-5fb8ca528091 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:41.874423 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:41.875049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:41.875330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:41.875634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] PATH_INFO: `/v3/users/9f97a682031645f1a03ac65e1dcf1d59` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:41.876775 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f97a682031645f1a03ac65e1dcf1d59)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:41.879356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:41.926274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f97a682031645f1a03ac65e1dcf1d59'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:41.926963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f97a682031645f1a03ac65e1dcf1d59'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:41.927605 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f97a682031645f1a03ac65e1dcf1d59'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:41.934448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f97a682031645f1a03ac65e1dcf1d59'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:41.942130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f97a682031645f1a03ac65e1dcf1d59'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:41.959680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1064/2128] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:41 2022] DELETE /identity/v3/users/9f97a682031645f1a03ac65e1dcf1d59 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:41.964140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:41.964579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:41.964720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:41.984060 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:41.985188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:42.042542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1065/2129] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:42.052494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.053889 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:42.109782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bab2a8-fd09-4792-bb98-61da87006a24 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.113386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8e5f7-baa0-468e-89f0-f608a8569587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.113990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e5f7-baa0-468e-89f0-f608a8569587 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.114267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e5f7-baa0-468e-89f0-f608a8569587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.114537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e5f7-baa0-468e-89f0-f608a8569587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.123784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8e5f7-baa0-468e-89f0-f608a8569587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.125203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8e5f7-baa0-468e-89f0-f608a8569587 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.135255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1065/2130] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:42.407340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.416661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:42.430893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5b648-1713-410c-b83f-bbc23e77d89e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.435626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.436288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.436566 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.436876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] PATH_INFO: `/v3/projects/8ac83bdb2fee4d8caa4a6c7ccbba05aa` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.438006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac83bdb2fee4d8caa4a6c7ccbba05aa)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.439967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.486801 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac83bdb2fee4d8caa4a6c7ccbba05aa', 'request_id': 'req-256413bb-49ec-4099-9a19-6c03067c74c8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:42.493438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac83bdb2fee4d8caa4a6c7ccbba05aa', 'request_id': 'req-256413bb-49ec-4099-9a19-6c03067c74c8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:42.500359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac83bdb2fee4d8caa4a6c7ccbba05aa', 'request_id': 'req-256413bb-49ec-4099-9a19-6c03067c74c8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:42.510953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] The token cache is being invalidate because project 8ac83bdb2fee4d8caa4a6c7ccbba05aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:42.511234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:42.511885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-256413bb-49ec-4099-9a19-6c03067c74c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:42.513350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1066/2131] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:42 2022] DELETE /identity/v3/projects/8ac83bdb2fee4d8caa4a6c7ccbba05aa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:42.518178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.518464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.518739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.539130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:42.540406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:42.556921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:42.599528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1066/2132] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:42.608080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.608368 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.608646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.629546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:42.630748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:42.638571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1067/2133] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:42.642569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.652548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282685a-55cc-4258-a5f4-ee741ad74f60 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.657026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.657617 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.657893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.658162 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.658805 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:16:42.659131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.661660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.673050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1067/2134] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:16:42.677373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.686450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da55503-fa7f-43c0-ac2e-572a53765ba7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.690996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.691613 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.691894 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.692164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.692797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.694684 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.720834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1068/2135] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:42 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:42.725690 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.735595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695555f3-8ba7-45c2-ae2f-e0895e1fafd4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.791202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.791840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.792121 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.792393 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.793016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.794926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.815273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1068/2136] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:42 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:42.819907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.830059 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384609-21f7-4dae-aa30-3b30d2353030 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.835432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.836037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.836311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.836580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.837216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.838921 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.849219 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1069/2137] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:42.853066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.862810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22a51-fed5-410a-a2c1-d3b825e036dc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.867680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e54715-ecee-4ef7-a0ab-13f1a4814570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.868269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e54715-ecee-4ef7-a0ab-13f1a4814570 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.868548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e54715-ecee-4ef7-a0ab-13f1a4814570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.868821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e54715-ecee-4ef7-a0ab-13f1a4814570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.869460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e54715-ecee-4ef7-a0ab-13f1a4814570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.871041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e54715-ecee-4ef7-a0ab-13f1a4814570 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.882018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1069/2138] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:16:42.885873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:42.896358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b772c-1bae-4c43-ab95-e272cf847504 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:42.900941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:42.901584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.901866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.902139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] PATH_INFO: `/v3/projects/c389780c754644c8832cd1de5afe30e8/users/75a89ca439ef458eaf4868dc1375df01/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.916755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c389780c754644c8832cd1de5afe30e8, user_id=75a89ca439ef458eaf4868dc1375df01, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:42.919452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:42.931841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1070/2139] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:42 2022] PUT /identity/v3/projects/c389780c754644c8832cd1de5afe30e8/users/75a89ca439ef458eaf4868dc1375df01/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:42.942703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:42.942991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:42.943266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:42.970571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] MFA Rules not processed for user `75a89ca439ef458eaf4868dc1375df01`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:42.971828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:43.036595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1070/2140] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:43.041389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.061545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:43.102688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de84f46-1cc1-4242-9266-46167254fbd7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.107312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.107938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.108213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.108481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.109104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.110996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.137754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1071/2141] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:43 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:43.141815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.152680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12445159-17b6-406f-9a26-c2e6eb34b959 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.165016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:43.209739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.210491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.210824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.211102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.211758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.213695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.236173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1071/2142] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:43 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:43.244200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.254396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3693e0-0d59-4a69-857e-a08703070205 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.258886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.259524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.259812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.260072 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.260705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.262373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.274620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1072/2143] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:16:43.278786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.289372 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9df1-ca61-4a5d-990e-9eacff283571 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.294293 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.294944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.295222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.295546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] PATH_INFO: `/v3/projects/4f02bb34a92d47629565a4e22d91f8df/users/820c5eef1564471a8eb3c786d4c1f03b/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.311010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f02bb34a92d47629565a4e22d91f8df, user_id=820c5eef1564471a8eb3c786d4c1f03b, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.313688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.328742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1072/2144] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:43 2022] PUT /identity/v3/projects/4f02bb34a92d47629565a4e22d91f8df/users/820c5eef1564471a8eb3c786d4c1f03b/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:43.333607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.344485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676436c-3b85-4f1d-b384-470208c2c298 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.401527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a3d2-5533-4a1b-b89a-1f59abb2748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.402134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a3d2-5533-4a1b-b89a-1f59abb2748f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.402418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a3d2-5533-4a1b-b89a-1f59abb2748f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.402694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a3d2-5533-4a1b-b89a-1f59abb2748f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.403338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3a3d2-5533-4a1b-b89a-1f59abb2748f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.404978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3a3d2-5533-4a1b-b89a-1f59abb2748f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.415632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1073/2145] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:43.419741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.429912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aa81a-0a2f-4b60-9fc0-909e6bca053b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.434494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.435132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.435455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.435729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] PATH_INFO: `/v3/projects/4f02bb34a92d47629565a4e22d91f8df/users/820c5eef1564471a8eb3c786d4c1f03b/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.437600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f02bb34a92d47629565a4e22d91f8df, user_id=820c5eef1564471a8eb3c786d4c1f03b, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.440274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.450159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1073/2146] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:43 2022] PUT /identity/v3/projects/4f02bb34a92d47629565a4e22d91f8df/users/820c5eef1564471a8eb3c786d4c1f03b/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:43.461792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.462081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.462355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.489787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] MFA Rules not processed for user `820c5eef1564471a8eb3c786d4c1f03b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:43.490987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:43.556850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1074/2147] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:43.567917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:43.569628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:43.627863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499bb07-16c4-4340-9bd0-81b491cf0ac0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:43.631964 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d317c74-aa22-4c39-9876-0233d9f09996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:43.632558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d317c74-aa22-4c39-9876-0233d9f09996 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:43.632838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d317c74-aa22-4c39-9876-0233d9f09996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:43.633109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d317c74-aa22-4c39-9876-0233d9f09996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:43.686737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d317c74-aa22-4c39-9876-0233d9f09996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:43.688194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d317c74-aa22-4c39-9876-0233d9f09996 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:43.697386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1074/2148] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:46.233807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:46.251223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:46.296327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e1fe-b412-4fb6-a186-c1cdc43bb226 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:46.301277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:46.301888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:46.302164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:46.302435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] PATH_INFO: `/v3/users/75a89ca439ef458eaf4868dc1375df01` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:46.303584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a89ca439ef458eaf4868dc1375df01)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:46.305504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:46.349326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a89ca439ef458eaf4868dc1375df01'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.349964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a89ca439ef458eaf4868dc1375df01'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.357163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a89ca439ef458eaf4868dc1375df01'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.357789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a89ca439ef458eaf4868dc1375df01'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.365564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a89ca439ef458eaf4868dc1375df01'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.393763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1075/2149] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:46 2022] DELETE /identity/v3/users/75a89ca439ef458eaf4868dc1375df01 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:46.398433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a7540-a820-4f4d-9419-ab3b2fdaed35 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:46.398892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a7540-a820-4f4d-9419-ab3b2fdaed35 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:46.399090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a7540-a820-4f4d-9419-ab3b2fdaed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:46.424344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d01a7540-a820-4f4d-9419-ab3b2fdaed35 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:46.426153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d01a7540-a820-4f4d-9419-ab3b2fdaed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:46.446630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:46.495119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1075/2150] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:46.506271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:46.507772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:46.546930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:46.547217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:46.547551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:46.566640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] MFA Rules not processed for user `52173fbead6e4b499a99a6979c9a9898`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:46.567871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:46.573238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fb654-e29a-47c6-b35b-cdbc878276d0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:46.576931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:46.577534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:46.577812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:46.578079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:46.587045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:46.588494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:46.598577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1076/2151] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:46.622319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1076/2152] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:16:46.865079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:46.868004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:46.885578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f9f6-b7e2-45a2-bd07-e6b2f35fe0d5 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:46.891095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:46.891997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:46.892341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:46.892684 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] PATH_INFO: `/v3/projects/c389780c754644c8832cd1de5afe30e8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:46.894114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c389780c754644c8832cd1de5afe30e8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:46.896130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:46.934982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:46.949987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c389780c754644c8832cd1de5afe30e8', 'request_id': 'req-98be3977-434c-416a-954f-3cdf0e7f0e72'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.956336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:46.956902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c389780c754644c8832cd1de5afe30e8', 'request_id': 'req-98be3977-434c-416a-954f-3cdf0e7f0e72'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.958700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:46.965332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c389780c754644c8832cd1de5afe30e8', 'request_id': 'req-98be3977-434c-416a-954f-3cdf0e7f0e72'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.975230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea215883-cafc-43f4-bd54-fd65dc1590c9 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:46.977198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] The token cache is being invalidate because project c389780c754644c8832cd1de5afe30e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:46.977485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.978146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:46.979929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1077/2153] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:46 2022] DELETE /identity/v3/projects/c389780c754644c8832cd1de5afe30e8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:46.983474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:46.984762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:47.025186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.025790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.026065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.026332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.028049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:47.037378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.038922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.049546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be3977-434c-416a-954f-3cdf0e7f0e72 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.051766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1077/2154] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:47.054744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.055376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.055686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.055957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] PATH_INFO: `/v3/users/820c5eef1564471a8eb3c786d4c1f03b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.057097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820c5eef1564471a8eb3c786d4c1f03b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.059021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.105122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820c5eef1564471a8eb3c786d4c1f03b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.105782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820c5eef1564471a8eb3c786d4c1f03b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.113290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820c5eef1564471a8eb3c786d4c1f03b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.113952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820c5eef1564471a8eb3c786d4c1f03b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.121910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820c5eef1564471a8eb3c786d4c1f03b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.140253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1078/2155] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:46 2022] DELETE /identity/v3/users/820c5eef1564471a8eb3c786d4c1f03b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:47.432465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:47.458061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eeb24-e08f-4b18-9062-e74bc060ff75 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.524723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:47.530146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:47.536745 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.537614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.538511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.538961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] PATH_INFO: `/v3/projects/4f02bb34a92d47629565a4e22d91f8df` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.543252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f02bb34a92d47629565a4e22d91f8df)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.545247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.570111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:47.593188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:47.610594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f02bb34a92d47629565a4e22d91f8df', 'request_id': 'req-641174a8-5434-4e52-9615-9913b453ac83'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.618716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f02bb34a92d47629565a4e22d91f8df', 'request_id': 'req-641174a8-5434-4e52-9615-9913b453ac83'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.622214 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5e4c-4da7-4396-b65d-14191161e806 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.625397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f02bb34a92d47629565a4e22d91f8df', 'request_id': 'req-641174a8-5434-4e52-9615-9913b453ac83'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.635774 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] The token cache is being invalidate because project 4f02bb34a92d47629565a4e22d91f8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:47.636063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.636693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-641174a8-5434-4e52-9615-9913b453ac83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:47.638399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1078/2156] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:47 2022] DELETE /identity/v3/projects/4f02bb34a92d47629565a4e22d91f8df => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:47.642969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.643254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.643558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.664403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:47.665753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:47.684249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.684958 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.685307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.685640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.687570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:47.699198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.700859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2f9b4-7ed5-4147-ae8c-259d008fe95e service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.712872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1079/2157] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:47.728618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1079/2158] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:16:47.736407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.736710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.736966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.756876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:47.758074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:47.765853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1080/2159] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:47.770565 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:47.780228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a307-236b-436d-9019-bf476d5e53fe None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.785443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.788559 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.789840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.790185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.790965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:16:47.791384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.793130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.805200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1080/2160] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:16:47.809681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:47.819447 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e564-82ee-4512-ab60-5371946d9b0f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.824321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.825032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.825311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.825582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.826256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.828207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.859770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1081/2161] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:47 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:47.860920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:47.872230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba6be9-a05b-4ba3-bfdf-328b45336270 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.936763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.937361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.937644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.937918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.938567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.940532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:47.966533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1081/2162] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:47 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:16:47.967696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:47.979437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fb18-776b-4dd4-89e1-fbc6456f53dd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:47.984883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:47.985603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:47.986080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:47.986458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:47.987246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:47.989120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.002949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1082/2163] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:48.004840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.014648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7731f-6d5d-4606-be68-e28406cab477 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.020022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961eaff8-2c37-48b4-89c7-b32ed445c23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.020622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961eaff8-2c37-48b4-89c7-b32ed445c23e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.020895 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961eaff8-2c37-48b4-89c7-b32ed445c23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.021164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961eaff8-2c37-48b4-89c7-b32ed445c23e admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.021823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961eaff8-2c37-48b4-89c7-b32ed445c23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.023382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961eaff8-2c37-48b4-89c7-b32ed445c23e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.034912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1082/2164] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:48.039016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.049431 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4f9f-7637-438d-acbc-3319c6e8c64d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.054735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.055400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.055726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.056035 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] PATH_INFO: `/v3/projects/0a052546833245028e6493405a173955/users/ebd5d9fc97ab4b73ac4229d9fee2ff1d/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.071975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a052546833245028e6493405a173955, user_id=ebd5d9fc97ab4b73ac4229d9fee2ff1d, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.074646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.087843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1083/2165] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:48 2022] PUT /identity/v3/projects/0a052546833245028e6493405a173955/users/ebd5d9fc97ab4b73ac4229d9fee2ff1d/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:48.098093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.098374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.098646 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.118025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:48.129137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] MFA Rules not processed for user `ebd5d9fc97ab4b73ac4229d9fee2ff1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:48.130505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:48.194884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1083/2166] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:48.199515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.265999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3a2f-05ef-4abc-ad2a-bb2133b437fb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.272288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.272903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.273326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.273717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.274357 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.276279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.292326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:48.312484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1084/2167] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:48 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:48.313860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.325791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c4b13-e972-4f98-baef-43cbb7feabd3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.392482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.393100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.393378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.393650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.394297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.396230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.419435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1084/2168] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:48 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:16:48.423456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.434006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05321a4-c544-42b2-a47e-75e94d86f19b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.438541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.439126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.439402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.439699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.440333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.442706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.453820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1085/2169] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:48.457780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.470216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305143c-ee67-49d4-8973-b8a49e68c520 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.474567 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d47e-8a92-4cbf-a995-081fc8c4920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.475155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d47e-8a92-4cbf-a995-081fc8c4920e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.475455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d47e-8a92-4cbf-a995-081fc8c4920e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.475728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d47e-8a92-4cbf-a995-081fc8c4920e admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.476364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109d47e-8a92-4cbf-a995-081fc8c4920e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.477927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109d47e-8a92-4cbf-a995-081fc8c4920e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.488376 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1085/2170] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:48.492730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.502466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43416bfe-ee67-47f5-8134-9754ea7889ef admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.507623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.508257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.508536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.508806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] PATH_INFO: `/v3/projects/a822143df21b44f984daa71cf37baeec/users/a5afbca7679e4a83af2d2b8fa45b8bd8/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.522870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a822143df21b44f984daa71cf37baeec, user_id=a5afbca7679e4a83af2d2b8fa45b8bd8, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.525555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.538482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1086/2171] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:48 2022] PUT /identity/v3/projects/a822143df21b44f984daa71cf37baeec/users/a5afbca7679e4a83af2d2b8fa45b8bd8/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:48.549647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.550508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.550896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.579342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] MFA Rules not processed for user `a5afbca7679e4a83af2d2b8fa45b8bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:48.580826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:48.666702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1086/2172] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:48.671349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.740549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063143ab-1104-4992-8cc0-ad7e372dbb4b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.745391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.745994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.746271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.746540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.747194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.749146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.780324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1087/2173] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:48 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:48.781689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.793888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59782cc-a45b-4f1e-8e9c-67cddfd263be None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.799667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:48.866612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.867506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.867786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.868086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.868720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.870641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.897585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1087/2174] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:48 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:48.899722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.911653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aada58c-f1d7-407b-82b4-8dcfe5ac85ed admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.916924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.917693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.918113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.919036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.919169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.920856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.936061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1088/2175] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:16:48.937438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:48.947904 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d669c9-7112-413a-b52c-b50cb329eac8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:48.953371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:48.954023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:48.954305 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:48.954580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] PATH_INFO: `/v3/projects/eda32a9feb754e6bb97eea1111c4ed4e/users/9fda25f9a1184628a92eb519cb4cbb6c/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:48.969636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda32a9feb754e6bb97eea1111c4ed4e, user_id=9fda25f9a1184628a92eb519cb4cbb6c, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:48.972971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:48.988394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1088/2176] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:48 2022] PUT /identity/v3/projects/eda32a9feb754e6bb97eea1111c4ed4e/users/9fda25f9a1184628a92eb519cb4cbb6c/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:48.992739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:49.003701 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c521f8-6962-43d0-b9d5-89485b15c611 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:49.066421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc5b12-a2d0-4036-8630-b43bace38537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:49.067031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc5b12-a2d0-4036-8630-b43bace38537 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:49.067310 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc5b12-a2d0-4036-8630-b43bace38537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:49.067618 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc5b12-a2d0-4036-8630-b43bace38537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:49.068264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc5b12-a2d0-4036-8630-b43bace38537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:49.069859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc5b12-a2d0-4036-8630-b43bace38537 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:49.079778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1089/2177] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:49.084033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:49.094319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d483cb-74ff-424d-a0e0-fb1bcdbc120e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:49.099953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:49.100608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:49.100887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:49.101251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] PATH_INFO: `/v3/projects/eda32a9feb754e6bb97eea1111c4ed4e/users/9fda25f9a1184628a92eb519cb4cbb6c/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:49.103112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda32a9feb754e6bb97eea1111c4ed4e, user_id=9fda25f9a1184628a92eb519cb4cbb6c, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:49.105984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:49.118036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1089/2178] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:49 2022] PUT /identity/v3/projects/eda32a9feb754e6bb97eea1111c4ed4e/users/9fda25f9a1184628a92eb519cb4cbb6c/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:49.128295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:49.128572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:49.128842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:49.157070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] MFA Rules not processed for user `9fda25f9a1184628a92eb519cb4cbb6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:49.158282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:49.175306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:49.230145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1090/2179] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:49.242967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:49.244809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:49.304452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3a90-b788-41d4-bc57-9b03913f05a5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:49.308024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:49.308631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:49.308912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:49.309186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:49.372516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:49.374276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:49.384825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1090/2180] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:49.606353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:49.609066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:49.727914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417c2d9-719a-4a09-bb73-9736d6dcd7cc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:49.732699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:49.733324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:49.733744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:49.734063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:49.739090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:49.755001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:49.758470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:49.774001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1091/2181] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:51.038679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:51.040427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:51.052721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de586-ab93-4951-a3d4-e75d7b897eb7 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:51.056321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:51.056920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:51.057199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:51.057469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:51.066788 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:51.115798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:51.117772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:51.126576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1091/2182] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:51.658963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:51.668755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34da96-6110-4676-bd4c-6fd920ddbcb4 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:51.672325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c6316-2f1b-48ed-8118-aa0374c3ed39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:51.672911 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c6316-2f1b-48ed-8118-aa0374c3ed39 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:51.673184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c6316-2f1b-48ed-8118-aa0374c3ed39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:51.673451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c6316-2f1b-48ed-8118-aa0374c3ed39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:51.682370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c6316-2f1b-48ed-8118-aa0374c3ed39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:51.683819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c6316-2f1b-48ed-8118-aa0374c3ed39 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:51.693095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1092/2183] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:16:55.551034 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:55.612756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215efb3d-2c3d-4ffa-a4eb-6ec4bf0a91f5 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:55.617530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:55.618148 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:55.618428 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:55.618697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] PATH_INFO: `/v3/users/ebd5d9fc97ab4b73ac4229d9fee2ff1d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:55.619834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd5d9fc97ab4b73ac4229d9fee2ff1d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:55.621807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:55.667092 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd5d9fc97ab4b73ac4229d9fee2ff1d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:55.667739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd5d9fc97ab4b73ac4229d9fee2ff1d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:55.668335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd5d9fc97ab4b73ac4229d9fee2ff1d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:55.675564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd5d9fc97ab4b73ac4229d9fee2ff1d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:55.682302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd5d9fc97ab4b73ac4229d9fee2ff1d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:55.698970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1092/2184] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:55 2022] DELETE /identity/v3/users/ebd5d9fc97ab4b73ac4229d9fee2ff1d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:16:55.703400 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:55.703925 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:55.704015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:55.722954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:55.724168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:55.782478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1093/2185] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:55.792543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:55.793955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:55.830893 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:55.859733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb860-4105-48f2-82b4-972707ff0e7b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:55.864179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:55.864861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:55.865229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:55.865538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:55.875281 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:55.876761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:55.886395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1093/2186] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:16:56.395339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:56.396897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:56.412181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a98fe-6734-4913-ad9e-806cc2786c59 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:56.417956 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:56.418599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:56.418878 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:56.419191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] PATH_INFO: `/v3/projects/0a052546833245028e6493405a173955` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:56.420328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a052546833245028e6493405a173955)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:56.422476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:56.464337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:56.472811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a052546833245028e6493405a173955', 'request_id': 'req-c408d625-2a3d-4e53-8547-6aa61f2b6605'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.479125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a052546833245028e6493405a173955', 'request_id': 'req-c408d625-2a3d-4e53-8547-6aa61f2b6605'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.486551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a052546833245028e6493405a173955', 'request_id': 'req-c408d625-2a3d-4e53-8547-6aa61f2b6605'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.497361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] The token cache is being invalidate because project 0a052546833245028e6493405a173955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:56.497647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.498273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.499783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1094/2187] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:56 2022] DELETE /identity/v3/projects/0a052546833245028e6493405a173955 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:56.504921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:56.515881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb75e-6914-44c8-8624-41558f0389ca service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:56.570063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:56.570683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:56.570868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:56.571134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] PATH_INFO: `/v3/users/a5afbca7679e4a83af2d2b8fa45b8bd8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:56.572544 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5afbca7679e4a83af2d2b8fa45b8bd8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:56.574482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:56.619482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5afbca7679e4a83af2d2b8fa45b8bd8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.620131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5afbca7679e4a83af2d2b8fa45b8bd8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.620770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5afbca7679e4a83af2d2b8fa45b8bd8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.628428 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5afbca7679e4a83af2d2b8fa45b8bd8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.635700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5afbca7679e4a83af2d2b8fa45b8bd8'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:56.652706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1094/2188] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:56 2022] DELETE /identity/v3/users/a5afbca7679e4a83af2d2b8fa45b8bd8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:56.934143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:56.935739 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:56.951463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408d625-2a3d-4e53-8547-6aa61f2b6605 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:57.014797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:57.015458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.015748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.016019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] PATH_INFO: `/v3/projects/a822143df21b44f984daa71cf37baeec` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.017150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a822143df21b44f984daa71cf37baeec)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:57.019110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:57.079276 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a822143df21b44f984daa71cf37baeec', 'request_id': 'req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.085892 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a822143df21b44f984daa71cf37baeec', 'request_id': 'req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.093685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a822143df21b44f984daa71cf37baeec', 'request_id': 'req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.106074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] The token cache is being invalidate because project a822143df21b44f984daa71cf37baeec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:57.106359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.106987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.108490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1095/2189] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:56 2022] DELETE /identity/v3/projects/a822143df21b44f984daa71cf37baeec => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:57.113045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:57.114597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:57.127388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f55f6-1a6c-4e39-bb29-77206eb9dc25 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:57.176795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:57.187516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:57.188149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.188437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.188713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] PATH_INFO: `/v3/users/9fda25f9a1184628a92eb519cb4cbb6c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.189833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fda25f9a1184628a92eb519cb4cbb6c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:57.191825 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:57.235881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fda25f9a1184628a92eb519cb4cbb6c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.236509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fda25f9a1184628a92eb519cb4cbb6c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.237752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fda25f9a1184628a92eb519cb4cbb6c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.244925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fda25f9a1184628a92eb519cb4cbb6c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.251759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5ecf7285-356b-49c2-9a4e-528d08ac766b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fda25f9a1184628a92eb519cb4cbb6c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.268394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1095/2190] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:16:57 2022] DELETE /identity/v3/users/9fda25f9a1184628a92eb519cb4cbb6c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:57.541824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:57.543269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:57.558316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbebd11-6a2e-40e3-acec-dbaf7f0fc8c7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:57.611372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:57.618437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:57.619075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.619722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.620215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] PATH_INFO: `/v3/projects/eda32a9feb754e6bb97eea1111c4ed4e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.621490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda32a9feb754e6bb97eea1111c4ed4e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:57.623836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:57.668863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda32a9feb754e6bb97eea1111c4ed4e', 'request_id': 'req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.675065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda32a9feb754e6bb97eea1111c4ed4e', 'request_id': 'req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.682845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda32a9feb754e6bb97eea1111c4ed4e', 'request_id': 'req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.693107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] The token cache is being invalidate because project eda32a9feb754e6bb97eea1111c4ed4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:16:57.693391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.694005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0d283ea4-d78b-4383-abe5-a4dfcb0a09e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:16:57.695496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1096/2191] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:16:57 2022] DELETE /identity/v3/projects/eda32a9feb754e6bb97eea1111c4ed4e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:57.700318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.700772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.700875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.720550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:57.722332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:57.779999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1096/2192] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:57.787962 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.788244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.788540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.808216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:57.809421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:57.817868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1097/2193] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:57.822030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:57.832271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2aad-a0a5-4ea1-89b5-e1eb200de596 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:57.836803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:57.837391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.837668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.837938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.838575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:16:57.838882 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:57.840499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:57.851720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1097/2194] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:16:57.855720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:57.865032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836a231-a491-48ab-a7cc-ebace55e85ce None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:57.869813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:57.870398 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.870676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.870945 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.871608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:57.873494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:57.901835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1098/2195] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:16:57 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:57.906172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:57.915870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c89a-6f83-4053-9d49-eabc66f0f7c4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:57.976723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:57.977343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:57.977609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:57.977880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:57.978545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:57.980504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:57.988520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:16:58.003575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1098/2196] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:16:57 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:16:58.007830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:58.017975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d950b8-fa49-4525-a29a-3148961365c3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:58.023162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:58.023778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:58.024057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:58.024326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:58.024989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:58.027049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:58.036732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1099/2197] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:58.040514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:58.050335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea062-5638-4c56-9ff0-b761613e5bc8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:58.054954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e9fd2-54ec-4698-9769-2ce39a7d1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:58.055580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9fd2-54ec-4698-9769-2ce39a7d1644 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:58.055859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9fd2-54ec-4698-9769-2ce39a7d1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:58.056131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9fd2-54ec-4698-9769-2ce39a7d1644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:58.056768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e9fd2-54ec-4698-9769-2ce39a7d1644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:58.058422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e9fd2-54ec-4698-9769-2ce39a7d1644 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:58.069506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1099/2198] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:16:58.073777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:58.084871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8555-1f2a-417b-85da-f393b0c2cdee admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:58.090498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:58.091154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:58.091445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:58.091731 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] PATH_INFO: `/v3/projects/b66b86910a70488e902b1cb1c9a34270/users/ade982e307c6440e9e40db070fac5412/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:58.106584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66b86910a70488e902b1cb1c9a34270, user_id=ade982e307c6440e9e40db070fac5412, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:58.109369 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:58.124996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1100/2199] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:16:58 2022] PUT /identity/v3/projects/b66b86910a70488e902b1cb1c9a34270/users/ade982e307c6440e9e40db070fac5412/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:16:58.138039 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:58.138409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:58.138696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:58.169327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] MFA Rules not processed for user `ade982e307c6440e9e40db070fac5412`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:16:58.171381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:58.248898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1100/2200] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:16:58.260486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:16:58.262152 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:16:58.317901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c826ff-0ecc-4fbf-ab5e-6da73b2feee4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:16:58.321535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:16:58.322131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:16:58.322410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:16:58.322682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:16:58.330693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:16:58.332148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:16:58.340675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1101/2201] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:16:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:03.238410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:03.240181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:03.316274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64263b26-7307-4405-93f7-4978f3b192fa None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:03.321687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:03.322290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:03.322570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:03.322844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:03.324571 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:03.336674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:03.338248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:03.349463 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1101/2202] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:17:05.472873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:05.474331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:05.504790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:05.536006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9611-80c3-40bf-869a-e5fbbe88c26f service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:05.539807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:05.540408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:05.540683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:05.540955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:05.542692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:05.607297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:05.608827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b466fca-2768-4af0-8ccf-d6db58d1cdea service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:05.619337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1102/2203] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:06.401263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:06.459208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:06.471143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65bf6a-0bcf-4e3a-8886-b242974f85ea service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:06.476306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:06.476938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:06.477216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:06.477491 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] PATH_INFO: `/v3/users/ade982e307c6440e9e40db070fac5412` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:06.478621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade982e307c6440e9e40db070fac5412)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:06.480593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:06.529934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade982e307c6440e9e40db070fac5412'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:06.530593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade982e307c6440e9e40db070fac5412'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:06.531196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade982e307c6440e9e40db070fac5412'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:06.538789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade982e307c6440e9e40db070fac5412'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:06.546017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade982e307c6440e9e40db070fac5412'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:06.563483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1102/2204] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:06 2022] DELETE /identity/v3/users/ade982e307c6440e9e40db070fac5412 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:06.568209 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:06.568763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:06.568763 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:06.590473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:06.591750 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:06.685896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1103/2205] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:06.695410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:06.696862 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:06.762441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863171e-b4fb-4e2a-a4d3-f46fd66a1284 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:06.766319 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:06.766921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:06.767196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:06.767489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:06.777348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:06.778769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:06.788846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1103/2206] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:07.060206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:07.061919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:07.076979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac9b20-539b-4020-bac5-0686fb62066d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:07.081457 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:07.082124 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:07.082394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:07.082662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] PATH_INFO: `/v3/projects/b66b86910a70488e902b1cb1c9a34270` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:07.083851 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66b86910a70488e902b1cb1c9a34270)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:07.085769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:07.138008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66b86910a70488e902b1cb1c9a34270', 'request_id': 'req-32d5a613-4904-4373-b98b-4ed8b988897a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:07.144969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66b86910a70488e902b1cb1c9a34270', 'request_id': 'req-32d5a613-4904-4373-b98b-4ed8b988897a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:07.155599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66b86910a70488e902b1cb1c9a34270', 'request_id': 'req-32d5a613-4904-4373-b98b-4ed8b988897a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:07.165942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:07.169540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] The token cache is being invalidate because project b66b86910a70488e902b1cb1c9a34270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:07.169860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:07.170716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:07.175502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1104/2207] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:07 2022] DELETE /identity/v3/projects/b66b86910a70488e902b1cb1c9a34270 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:07.194386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:07.199585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:07.290359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a40b-5874-4331-8d2d-17d08d4eca32 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:07.302476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:07.373315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:07.373939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:07.374216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:07.374485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:07.376328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:07.391483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:07.393060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:07.414291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1104/2208] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:17.400819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=28 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:19.440311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:19.441766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:19.499036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a613-4904-4373-b98b-4ed8b988897a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:19.502549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:19.503143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:19.503442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:19.503792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:19.505545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:19.566107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:19.567813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:19.578997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1105/2209] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:23.125335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.126846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:23.140663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a7b5-f20b-420e-a10d-df6482d26843 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.145718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.146366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.146641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.146913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] PATH_INFO: `/v3/users/1c602e0f9e0e4e479a6a3c32febdb14b` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.148041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c602e0f9e0e4e479a6a3c32febdb14b)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.149954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.195253 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c602e0f9e0e4e479a6a3c32febdb14b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.195860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c602e0f9e0e4e479a6a3c32febdb14b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.196471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c602e0f9e0e4e479a6a3c32febdb14b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.203174 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c602e0f9e0e4e479a6a3c32febdb14b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.210720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ff66fcba-4151-48ea-8f97-02aafebf38ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c602e0f9e0e4e479a6a3c32febdb14b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.227624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1105/2210] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:23 2022] DELETE /identity/v3/users/1c602e0f9e0e4e479a6a3c32febdb14b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:23.424542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.434807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1f65b-2bba-4c4e-b551-e1eccc5a1011 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.504354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.504989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.505271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.505565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] PATH_INFO: `/v3/projects/bf094dc7d5f1453ba9112d1a1d3f0d9c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.506829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf094dc7d5f1453ba9112d1a1d3f0d9c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.508814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.562237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf094dc7d5f1453ba9112d1a1d3f0d9c', 'request_id': 'req-d345cf42-4449-4970-8a7d-8e14686b7280'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.564063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:23.569891 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf094dc7d5f1453ba9112d1a1d3f0d9c', 'request_id': 'req-d345cf42-4449-4970-8a7d-8e14686b7280'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.578233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf094dc7d5f1453ba9112d1a1d3f0d9c', 'request_id': 'req-d345cf42-4449-4970-8a7d-8e14686b7280'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.588937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] The token cache is being invalidate because project bf094dc7d5f1453ba9112d1a1d3f0d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:23.589222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.589840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d345cf42-4449-4970-8a7d-8e14686b7280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:23.591327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1106/2211] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:23 2022] DELETE /identity/v3/projects/bf094dc7d5f1453ba9112d1a1d3f0d9c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:23.597381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.597677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.597968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.617828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:23.619066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:23.675228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1106/2212] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:23.683196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.683517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.683797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.705074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:23.706205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:23.714113 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1107/2213] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:23.718210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.727885 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8025e03-9c8e-4362-a6ea-c65160ace628 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.732724 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.733334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.733652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.733866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.734616 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:17:23.735069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.737172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.750783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1107/2214] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:17:23.755748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.765070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38ac5f-09d9-4b85-9fdf-f6f0df567034 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.769797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.770391 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.770667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.770937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.771616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.773530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.800849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1108/2215] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:23 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:23.804980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.814743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf3b59-a771-48dc-8cd7-215d64ad38c8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.868781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.869403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.869679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.869952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.870588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.872519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.880905 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:23.894334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1108/2216] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:23 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:23.898507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.908777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59263a6-01b8-44ee-b808-c167629b2048 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.914015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.914610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.914889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.915156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.915837 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.917456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.929213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1109/2217] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:23.932687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.942866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e7dab-d6bd-4a8f-9193-97bc3bddf313 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.947696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536cd10-b95e-498c-bf7c-4a25a71ef6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.948296 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536cd10-b95e-498c-bf7c-4a25a71ef6ac admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.948573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536cd10-b95e-498c-bf7c-4a25a71ef6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.948842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536cd10-b95e-498c-bf7c-4a25a71ef6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:23.949599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536cd10-b95e-498c-bf7c-4a25a71ef6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:23.951216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536cd10-b95e-498c-bf7c-4a25a71ef6ac admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:23.962118 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1109/2218] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:23.966303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:23.978701 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08362318-2795-4630-8455-3ee8a86b75ba admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:23.984523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:23.985187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:23.985472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:23.985744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] PATH_INFO: `/v3/projects/c9aac62e83894a6b9be41abc043dd492/users/7662654c25e54d87a38fe3d3d060ac7b/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:24.001434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9aac62e83894a6b9be41abc043dd492, user_id=7662654c25e54d87a38fe3d3d060ac7b, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:24.004193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:24.017827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1110/2219] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:23 2022] PUT /identity/v3/projects/c9aac62e83894a6b9be41abc043dd492/users/7662654c25e54d87a38fe3d3d060ac7b/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:24.023154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e075e-5d46-46e4-951d-3fc787c7a01c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:24.023369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e075e-5d46-46e4-951d-3fc787c7a01c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:24.023681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e075e-5d46-46e4-951d-3fc787c7a01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:24.044686 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-403e075e-5d46-46e4-951d-3fc787c7a01c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:24.045891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-403e075e-5d46-46e4-951d-3fc787c7a01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:24.102635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1110/2220] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:24.112359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:24.114696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:24.169880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcab94-7680-4a3a-af79-a112b066406b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:24.173488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:24.174084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:24.174360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:24.174627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:24.184045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:24.185466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:24.194773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1111/2221] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:24.819601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:24.820527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:24.820798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:24.849108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] MFA Rules not processed for user `7662654c25e54d87a38fe3d3d060ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:24.850324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:24.916517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1111/2222] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:17:24.921437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:24.931436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427286df-071d-4b3c-8109-577b3607adb3 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:24.936081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:24.936717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:24.937021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:24.937286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:24.937959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:24.939910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:24.968855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1112/2223] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:24 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:24.973015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:24.984467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e4983-d198-4e06-984e-9bbd38fb0a9c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:25.041214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:25.041821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:25.042097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:25.042363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:25.043020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:25.044957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:25.060841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:25.069254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1112/2224] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:24 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:25.072835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:25.083574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c536a-cb93-4453-a222-81da34a5cdbf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:25.088608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:25.089294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:25.089573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:25.089844 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:25.090486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:25.092163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:25.103371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1113/2225] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:25.107516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:25.118175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21942aa2-a1ad-4e50-bf12-5c7d1f139426 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:25.122859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:25.123546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:25.123831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:25.124105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] PATH_INFO: `/v3/projects/213cb8c4f6ef498da6f301f617fb5200/users/9c79ef6da806494680da18572f95d476/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:25.139301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213cb8c4f6ef498da6f301f617fb5200, user_id=9c79ef6da806494680da18572f95d476, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:25.141987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:25.155101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1113/2226] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:25 2022] PUT /identity/v3/projects/213cb8c4f6ef498da6f301f617fb5200/users/9c79ef6da806494680da18572f95d476/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:25.159151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:25.169860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf876-04a4-4252-b8de-331d99df7661 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:25.213360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:25.227092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8c83-d32a-4fdc-924c-f4f8e4310c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:25.227717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8c83-d32a-4fdc-924c-f4f8e4310c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:25.227995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8c83-d32a-4fdc-924c-f4f8e4310c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:25.228264 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8c83-d32a-4fdc-924c-f4f8e4310c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:25.228896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba8c83-d32a-4fdc-924c-f4f8e4310c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:25.230478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba8c83-d32a-4fdc-924c-f4f8e4310c38 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:25.240486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1114/2227] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:25.245052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:25.255743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc79d2-12b9-481e-9f11-bbfeb9330d44 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:25.260286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:25.260956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:25.261235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:25.261507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] PATH_INFO: `/v3/projects/213cb8c4f6ef498da6f301f617fb5200/users/9c79ef6da806494680da18572f95d476/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:25.263398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] RBAC: Authorizing `identity:create_grant(project_id=213cb8c4f6ef498da6f301f617fb5200, user_id=9c79ef6da806494680da18572f95d476, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:25.266078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:25.276086 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1114/2228] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:25 2022] PUT /identity/v3/projects/213cb8c4f6ef498da6f301f617fb5200/users/9c79ef6da806494680da18572f95d476/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:25.893905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:25.894775 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:25.895140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:25.924330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] MFA Rules not processed for user `9c79ef6da806494680da18572f95d476`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:25.925788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:25.994870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1115/2229] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:26.451431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:26.453196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:26.529913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ead96-fc8d-4591-85d1-0454cea6971c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:26.535017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:26.535661 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:26.535942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:26.536217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:26.537956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:26.549483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:26.551614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:26.561339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1115/2230] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:27.015136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:27.077025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed807129-042e-4bbe-b775-57ebeba9d5bb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:27.081549 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:27.082147 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:27.082432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:27.082756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] PATH_INFO: `/v3/users/7662654c25e54d87a38fe3d3d060ac7b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:27.083960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7662654c25e54d87a38fe3d3d060ac7b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:27.085877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:27.131507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7662654c25e54d87a38fe3d3d060ac7b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.132151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7662654c25e54d87a38fe3d3d060ac7b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.138951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7662654c25e54d87a38fe3d3d060ac7b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.139596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7662654c25e54d87a38fe3d3d060ac7b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.146738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7662654c25e54d87a38fe3d3d060ac7b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.151931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:27.165668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1116/2231] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:27 2022] DELETE /identity/v3/users/7662654c25e54d87a38fe3d3d060ac7b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:27.335805 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:27.347469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13befc-6ccc-45c0-bae9-6c1db33de761 service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:27.404735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:27.405303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:27.405580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:27.405845 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] PATH_INFO: `/v3/projects/c9aac62e83894a6b9be41abc043dd492` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:27.406961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9aac62e83894a6b9be41abc043dd492)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:27.408892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:27.465147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9aac62e83894a6b9be41abc043dd492', 'request_id': 'req-758599df-92d9-47a6-a9c3-6ee399e0b2b0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.467823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:27.474195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9aac62e83894a6b9be41abc043dd492', 'request_id': 'req-758599df-92d9-47a6-a9c3-6ee399e0b2b0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.482204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9aac62e83894a6b9be41abc043dd492', 'request_id': 'req-758599df-92d9-47a6-a9c3-6ee399e0b2b0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.498959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] The token cache is being invalidate because project c9aac62e83894a6b9be41abc043dd492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:27.499249 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.499957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.501498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1116/2232] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:27 2022] DELETE /identity/v3/projects/c9aac62e83894a6b9be41abc043dd492 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:27.506208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:27.507830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:27.520783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684f53-7cdd-40f4-83d3-a66541c9e99d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:27.575927 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:27.576552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:27.576836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:27.577112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] PATH_INFO: `/v3/users/9c79ef6da806494680da18572f95d476` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:27.578462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c79ef6da806494680da18572f95d476)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:27.580440 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:27.625721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c79ef6da806494680da18572f95d476'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.626359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c79ef6da806494680da18572f95d476'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.633782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c79ef6da806494680da18572f95d476'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.634395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c79ef6da806494680da18572f95d476'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.641798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8a370710-a0da-45f6-9375-652412de067c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c79ef6da806494680da18572f95d476'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.658754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1117/2233] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:27 2022] DELETE /identity/v3/users/9c79ef6da806494680da18572f95d476 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:27.833963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:27.835383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:27.848654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758599df-92d9-47a6-a9c3-6ee399e0b2b0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:27.904828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:27.905451 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:27.905726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:27.905992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] PATH_INFO: `/v3/projects/213cb8c4f6ef498da6f301f617fb5200` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:27.907090 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=213cb8c4f6ef498da6f301f617fb5200)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:27.909096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:27.955982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213cb8c4f6ef498da6f301f617fb5200', 'request_id': 'req-44b3df23-b1fc-4339-9ea8-5669f4613cba'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.962827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213cb8c4f6ef498da6f301f617fb5200', 'request_id': 'req-44b3df23-b1fc-4339-9ea8-5669f4613cba'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.969134 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213cb8c4f6ef498da6f301f617fb5200', 'request_id': 'req-44b3df23-b1fc-4339-9ea8-5669f4613cba'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.980368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] The token cache is being invalidate because project 213cb8c4f6ef498da6f301f617fb5200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:27.980657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.981327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-44b3df23-b1fc-4339-9ea8-5669f4613cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:27.982930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1117/2234] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:27 2022] DELETE /identity/v3/projects/213cb8c4f6ef498da6f301f617fb5200 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:27.988425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:27.988713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:27.988986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.009450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:28.010641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:28.068422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1118/2235] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:28.075808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.076096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.076374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.097784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:28.098969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:28.106630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1118/2236] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:28.110645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.120403 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677727d-609d-40fa-b85c-e5ca66c78b69 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.124827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.125412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.125723 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.125998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.126636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:17:28.126929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.128542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.129974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:28.141359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1119/2237] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:17:28.145955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.154958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e5ef7-2466-4b53-9347-96ee50ae1b28 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.160301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.160842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.161119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.161389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.162028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.163948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.189507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1119/2238] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:28.193580 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.203385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d125d8-393a-412a-9021-181b673f0ed5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.258676 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.259292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.259592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.259869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.260501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.262423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.284182 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1120/2239] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:17:28.288939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.298876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9de9a-e4d7-462a-b5b9-4a143de88fdc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.303461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.304063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.304323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.304592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.305227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.306809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.316951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1120/2240] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:28.320650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.330468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063e0b9-fd24-4cc3-b59b-9bc621b5cc3a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.335623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627da83-2d23-429f-bdf6-8a24648c7579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.336206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627da83-2d23-429f-bdf6-8a24648c7579 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.336487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627da83-2d23-429f-bdf6-8a24648c7579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.336761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627da83-2d23-429f-bdf6-8a24648c7579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.337419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627da83-2d23-429f-bdf6-8a24648c7579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.339006 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627da83-2d23-429f-bdf6-8a24648c7579 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.349996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1121/2241] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:28.353827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.363647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dff2b8-d15d-49b3-b7fc-0940cf42061b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.369823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.370471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.370751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.371025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] PATH_INFO: `/v3/projects/e4fc63b5b6a14b2993c5bdf22939fb0f/users/19048a0e008148988de93f4089c894eb/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.385558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fc63b5b6a14b2993c5bdf22939fb0f, user_id=19048a0e008148988de93f4089c894eb, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.388251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.400840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1121/2242] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:28 2022] PUT /identity/v3/projects/e4fc63b5b6a14b2993c5bdf22939fb0f/users/19048a0e008148988de93f4089c894eb/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:28.412826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.413052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.413354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.441556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] MFA Rules not processed for user `19048a0e008148988de93f4089c894eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:28.442753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:28.506738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1122/2243] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:28.511165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.520628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:28.576141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96b72a-6384-43b1-8749-a18dde489428 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.580698 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.581292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.582206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.582473 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.583141 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.585052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.610192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1122/2244] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:17:28.614333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.625505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869244f-3f22-480e-a043-fb21e71a5cbc None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.681405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.681933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.682205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.682473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.683095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.685033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.694234 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:28.707472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1123/2245] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:17:28.711373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.721475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b2fee-3ff1-4076-a1ea-c69ce1aa3206 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.726044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.726631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.726907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.727188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.727859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.729429 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.739747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1123/2246] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:17:28.743898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.754046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2639b-81c3-49fd-aabe-03c028c1b9d5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.758561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.759188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.759492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.759761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] PATH_INFO: `/v3/projects/ce4ab501b74344cba710b5b21e9aec9d/users/83d1a9aaad264777be74275a506bd82a/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.774787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4ab501b74344cba710b5b21e9aec9d, user_id=83d1a9aaad264777be74275a506bd82a, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.777528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.791065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1124/2247] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:28 2022] PUT /identity/v3/projects/ce4ab501b74344cba710b5b21e9aec9d/users/83d1a9aaad264777be74275a506bd82a/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:28.795012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.805499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091728-3a69-4cae-a053-2b97097c1dab admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.864625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbd657-7e00-47cc-b2e3-5e6ec57d5bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.865229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbd657-7e00-47cc-b2e3-5e6ec57d5bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.865508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbd657-7e00-47cc-b2e3-5e6ec57d5bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.865777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbd657-7e00-47cc-b2e3-5e6ec57d5bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.866694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbd657-7e00-47cc-b2e3-5e6ec57d5bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.868322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbd657-7e00-47cc-b2e3-5e6ec57d5bc6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.877772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1124/2248] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:28.882679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:28.893207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746306a-5beb-47f8-ba64-a51aaee7d317 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:28.897964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:28.898605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.898884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.899158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] PATH_INFO: `/v3/projects/ce4ab501b74344cba710b5b21e9aec9d/users/83d1a9aaad264777be74275a506bd82a/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.901125 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4ab501b74344cba710b5b21e9aec9d, user_id=83d1a9aaad264777be74275a506bd82a, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:28.903824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:28.913918 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1125/2249] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:28 2022] PUT /identity/v3/projects/ce4ab501b74344cba710b5b21e9aec9d/users/83d1a9aaad264777be74275a506bd82a/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:28.923720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:28.924009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:28.924280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:28.952823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] MFA Rules not processed for user `83d1a9aaad264777be74275a506bd82a`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:28.953993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:29.020239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1125/2250] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:29.030777 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:29.032532 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:29.089621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8ba71-ee28-4733-95fb-627a9e22281a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:29.093288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be661-ae0a-420b-92da-e9673e5e1c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:29.093883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be661-ae0a-420b-92da-e9673e5e1c1a service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:29.094158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be661-ae0a-420b-92da-e9673e5e1c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:29.094427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be661-ae0a-420b-92da-e9673e5e1c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:29.103174 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942be661-ae0a-420b-92da-e9673e5e1c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:29.104630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942be661-ae0a-420b-92da-e9673e5e1c1a service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:29.114314 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1126/2251] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:30.434583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:30.438820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:30.496670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f202ff-b8bf-4ea7-9fd1-e0ac2f596abd None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:30.501106 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:30.501754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:30.502031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:30.502301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] PATH_INFO: `/v3/users/19048a0e008148988de93f4089c894eb` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:30.503437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19048a0e008148988de93f4089c894eb)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:30.505939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:30.549981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19048a0e008148988de93f4089c894eb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:30.550625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19048a0e008148988de93f4089c894eb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:30.551727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19048a0e008148988de93f4089c894eb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:30.558425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19048a0e008148988de93f4089c894eb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:30.565722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19048a0e008148988de93f4089c894eb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:30.583108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1126/2252] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:30 2022] DELETE /identity/v3/users/19048a0e008148988de93f4089c894eb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:30.587655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:30.588123 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:30.588211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:30.606531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:30.607751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:30.669986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1127/2253] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:30.680212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:30.681624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:30.736749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39d5e-0a41-408b-9f8c-b1fb03a0d45f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:30.741336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:30.741928 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:30.742202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:30.742467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:30.751240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:30.752621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:30.761728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1127/2254] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:31.027178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:31.028644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:31.041649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303959d9-20ad-4cac-9888-17088de9de8f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:31.046291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:31.046926 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.047203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.047903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] PATH_INFO: `/v3/projects/e4fc63b5b6a14b2993c5bdf22939fb0f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.048687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fc63b5b6a14b2993c5bdf22939fb0f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:31.050610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:31.098510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fc63b5b6a14b2993c5bdf22939fb0f', 'request_id': 'req-32821693-cfb5-451e-bf60-5580e93bfd2e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.100389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:31.105322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fc63b5b6a14b2993c5bdf22939fb0f', 'request_id': 'req-32821693-cfb5-451e-bf60-5580e93bfd2e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.112480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fc63b5b6a14b2993c5bdf22939fb0f', 'request_id': 'req-32821693-cfb5-451e-bf60-5580e93bfd2e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.122791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] The token cache is being invalidate because project e4fc63b5b6a14b2993c5bdf22939fb0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:31.123073 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.123697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.125172 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1128/2255] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:31 2022] DELETE /identity/v3/projects/e4fc63b5b6a14b2993c5bdf22939fb0f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:31.129120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:31.138441 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a56024-911f-48ca-8d01-9953ac67d578 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:31.157675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:31.195272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:31.195922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.196201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.196464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] PATH_INFO: `/v3/users/83d1a9aaad264777be74275a506bd82a` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.197573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d1a9aaad264777be74275a506bd82a)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:31.199510 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:31.245360 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d1a9aaad264777be74275a506bd82a'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.245995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d1a9aaad264777be74275a506bd82a'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.246589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d1a9aaad264777be74275a506bd82a'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.253321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d1a9aaad264777be74275a506bd82a'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.260119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d26cdb1f-f421-494a-a03c-f330b25b075f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d1a9aaad264777be74275a506bd82a'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.277601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1128/2256] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:31 2022] DELETE /identity/v3/users/83d1a9aaad264777be74275a506bd82a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:31.539542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:31.541078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:31.558972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821693-cfb5-451e-bf60-5580e93bfd2e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:31.615743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:31.616617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.616922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.617190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] PATH_INFO: `/v3/projects/ce4ab501b74344cba710b5b21e9aec9d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.618297 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4ab501b74344cba710b5b21e9aec9d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:31.620285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:31.665609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4ab501b74344cba710b5b21e9aec9d', 'request_id': 'req-2146a727-d900-47cc-be70-8cd9f02e3050'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.671646 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4ab501b74344cba710b5b21e9aec9d', 'request_id': 'req-2146a727-d900-47cc-be70-8cd9f02e3050'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.679390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4ab501b74344cba710b5b21e9aec9d', 'request_id': 'req-2146a727-d900-47cc-be70-8cd9f02e3050'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.690320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] The token cache is being invalidate because project ce4ab501b74344cba710b5b21e9aec9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:31.690612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.691825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-2146a727-d900-47cc-be70-8cd9f02e3050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:31.693341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1129/2257] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:31 2022] DELETE /identity/v3/projects/ce4ab501b74344cba710b5b21e9aec9d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:31.698971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.699493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.699587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.719302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:31.720538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:31.777250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1129/2258] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:31.784963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.785220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.785489 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.805003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:31.806179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:31.813895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1130/2259] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:31.818620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:31.828294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7f61c-4f26-4fe6-9029-afc9f7025da4 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:31.832859 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:31.833455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.833726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.834003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.834645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:17:31.834947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:31.836695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:31.848175 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1130/2260] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:17:31.852230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:31.861223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8e3a3-e5fb-460c-bc52-d173bdf7b04e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:31.866615 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:31.867198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.867505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.867773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.868405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:31.870302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:31.903253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1131/2261] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:31 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:31.907543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:31.917347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0942fa-e660-462b-b8af-3ff0ca4be54e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:31.948476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:31.977172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:31.977786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:31.978062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:31.978332 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:31.978965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:31.980870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.007974 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1131/2262] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:31 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:32.012021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.022286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a8eef-3542-43f7-b6fa-a44c9e70eb58 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.026797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.027435 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.027715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.027979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.028610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.030195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.039987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1132/2263] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:32.043656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.053652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba585-ee7b-4b65-ba88-0b6c941eacff admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.058164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62ca71-598c-4636-b9cf-b255a9d6a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.058749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62ca71-598c-4636-b9cf-b255a9d6a262 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.059020 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62ca71-598c-4636-b9cf-b255a9d6a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.059286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62ca71-598c-4636-b9cf-b255a9d6a262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.059952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62ca71-598c-4636-b9cf-b255a9d6a262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.061511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62ca71-598c-4636-b9cf-b255a9d6a262 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.071773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1132/2264] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:32.075659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.085332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990696a-e54e-49e2-9fff-5bcd9d5114f2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.089830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.090442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.090710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.090978 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] PATH_INFO: `/v3/projects/b6fe6b22fdc34d0299226aab76a22b30/users/2cf25e150dcd4ce0bfdf18021f2abf5d/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.105173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fe6b22fdc34d0299226aab76a22b30, user_id=2cf25e150dcd4ce0bfdf18021f2abf5d, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.107869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.120483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1133/2265] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:32 2022] PUT /identity/v3/projects/b6fe6b22fdc34d0299226aab76a22b30/users/2cf25e150dcd4ce0bfdf18021f2abf5d/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:32.132316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.132601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.132874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.159235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] MFA Rules not processed for user `2cf25e150dcd4ce0bfdf18021f2abf5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:32.160981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:32.221368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1133/2266] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:32 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:32.226312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.235117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:32.290657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79889667-49b9-4082-a5b5-f67dc810b920 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.295112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.295736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.296010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.296280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.296901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.298782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.325097 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1134/2267] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:32 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:32.329167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.339442 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc884cd-635c-4222-99d4-d98faf241d4e None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.394900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.395515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.395791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.396058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.396689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.398605 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.420036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1134/2268] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:32 2022] POST /identity/v3/users => generated 306 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:32.424001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.434686 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0a65b-a007-4010-99de-448032785933 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.439253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.439879 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.440158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.440427 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.441068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.442710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.453143 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1135/2269] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:32.457045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.466758 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445d2ae-914e-42ed-8fff-fd5a950dbd3c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.471247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.471915 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.472189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.472981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] PATH_INFO: `/v3/projects/df8b05dd172d4226af0d7448ff330302/users/37d371fb7b98414dbb75dfc32829966d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.487167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8b05dd172d4226af0d7448ff330302, user_id=37d371fb7b98414dbb75dfc32829966d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.490414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.502637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1135/2270] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:32 2022] PUT /identity/v3/projects/df8b05dd172d4226af0d7448ff330302/users/37d371fb7b98414dbb75dfc32829966d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:32.506612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.516390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeebdd8-dc0a-4ce7-947b-6a4eb487c0b9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.572197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43632902-ac0f-4a40-96b4-94ee2f935a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.572757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632902-ac0f-4a40-96b4-94ee2f935a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.573038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632902-ac0f-4a40-96b4-94ee2f935a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.573313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632902-ac0f-4a40-96b4-94ee2f935a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.574016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632902-ac0f-4a40-96b4-94ee2f935a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.575677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632902-ac0f-4a40-96b4-94ee2f935a8c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.585871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1136/2271] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:32.590381 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.596947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:32.601153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafa716-af2e-4c43-a838-4b8466cb9ca8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.606695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.607345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.607746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.608021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] PATH_INFO: `/v3/projects/df8b05dd172d4226af0d7448ff330302/users/37d371fb7b98414dbb75dfc32829966d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.609921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8b05dd172d4226af0d7448ff330302, user_id=37d371fb7b98414dbb75dfc32829966d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.612624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.623123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1136/2272] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:32 2022] PUT /identity/v3/projects/df8b05dd172d4226af0d7448ff330302/users/37d371fb7b98414dbb75dfc32829966d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:32.633322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.633797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.633900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.662824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] MFA Rules not processed for user `37d371fb7b98414dbb75dfc32829966d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:32.664090 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:32.731227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1137/2273] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:32 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:32.742234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.743985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:32.801715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f851-909e-45a7-8ecf-63ef45994e22 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.805355 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.805957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.806239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.806512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.815280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.816731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.825966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1137/2274] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:32.901683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:32.906193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:32.912972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c09ec-0d98-4317-b4a8-ad83642c1cb7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:32.916801 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55078fa-00f3-471b-b745-6f50e9339e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:32.917394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55078fa-00f3-471b-b745-6f50e9339e67 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:32.917674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55078fa-00f3-471b-b745-6f50e9339e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:32.917946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55078fa-00f3-471b-b745-6f50e9339e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:32.973825 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55078fa-00f3-471b-b745-6f50e9339e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:32.975244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55078fa-00f3-471b-b745-6f50e9339e67 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:32.983883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1138/2275] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:32 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:37.025189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:37.085888 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f12ac-89b2-4e76-a2a1-67ed36719875 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:37.090412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:37.091027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:37.091307 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:37.091611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] PATH_INFO: `/v3/users/2cf25e150dcd4ce0bfdf18021f2abf5d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:37.092768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf25e150dcd4ce0bfdf18021f2abf5d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:37.094689 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:37.139500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf25e150dcd4ce0bfdf18021f2abf5d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.140097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf25e150dcd4ce0bfdf18021f2abf5d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.140702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf25e150dcd4ce0bfdf18021f2abf5d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.148043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf25e150dcd4ce0bfdf18021f2abf5d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.154952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf25e150dcd4ce0bfdf18021f2abf5d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.172253 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1138/2276] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:37 2022] DELETE /identity/v3/users/2cf25e150dcd4ce0bfdf18021f2abf5d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:17:37.176898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a3597-372c-4321-b823-97292c27683f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:37.177372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a3597-372c-4321-b823-97292c27683f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:37.177462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a3597-372c-4321-b823-97292c27683f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:37.197134 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-894a3597-372c-4321-b823-97292c27683f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:37.198331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-894a3597-372c-4321-b823-97292c27683f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:37.257635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1139/2277] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:37.267522 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:37.268976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:37.283969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:37.327311 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fed2-9bae-47ed-bc62-46af32dcfed1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:37.330952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:37.332109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:37.332386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:37.332659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:37.341410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:37.342832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:37.352772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1139/2278] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:37.619286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a3597-372c-4321-b823-97292c27683f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:37.620737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-894a3597-372c-4321-b823-97292c27683f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:37.633170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a3597-372c-4321-b823-97292c27683f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:37.638721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:37.639443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:37.639718 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:37.639984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] PATH_INFO: `/v3/projects/b6fe6b22fdc34d0299226aab76a22b30` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:37.641255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fe6b22fdc34d0299226aab76a22b30)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:37.644702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:37.689887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fe6b22fdc34d0299226aab76a22b30', 'request_id': 'req-9e0169a2-0406-4714-a778-618baf24978f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.696845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fe6b22fdc34d0299226aab76a22b30', 'request_id': 'req-9e0169a2-0406-4714-a778-618baf24978f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.704285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fe6b22fdc34d0299226aab76a22b30', 'request_id': 'req-9e0169a2-0406-4714-a778-618baf24978f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.715015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] The token cache is being invalidate because project b6fe6b22fdc34d0299226aab76a22b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:37.715300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.715974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.717450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1140/2279] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:37 2022] DELETE /identity/v3/projects/b6fe6b22fdc34d0299226aab76a22b30 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:37.721418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:37.731063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ff5e3-d0e0-484b-a8a1-ccc581c90b84 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:37.786275 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:37.786891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:37.787163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:37.787443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] PATH_INFO: `/v3/users/37d371fb7b98414dbb75dfc32829966d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:37.788562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d371fb7b98414dbb75dfc32829966d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:37.790457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:37.834854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d371fb7b98414dbb75dfc32829966d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.835520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d371fb7b98414dbb75dfc32829966d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.836115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d371fb7b98414dbb75dfc32829966d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.842736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d371fb7b98414dbb75dfc32829966d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.849873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-18efeafe-abac-40c3-ade4-6cc4680a4815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d371fb7b98414dbb75dfc32829966d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:37.867143 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1140/2280] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:37 2022] DELETE /identity/v3/users/37d371fb7b98414dbb75dfc32829966d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:38.143304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.144768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:38.157451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0169a2-0406-4714-a778-618baf24978f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.178792 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:38.217000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.217561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.217842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.218116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] PATH_INFO: `/v3/projects/df8b05dd172d4226af0d7448ff330302` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.219230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8b05dd172d4226af0d7448ff330302)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.221218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.267319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8b05dd172d4226af0d7448ff330302', 'request_id': 'req-0fa096aa-d79d-425e-a5c3-73899789b571'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:38.273486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8b05dd172d4226af0d7448ff330302', 'request_id': 'req-0fa096aa-d79d-425e-a5c3-73899789b571'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:38.281202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8b05dd172d4226af0d7448ff330302', 'request_id': 'req-0fa096aa-d79d-425e-a5c3-73899789b571'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:38.293472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] The token cache is being invalidate because project df8b05dd172d4226af0d7448ff330302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:38.293757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:38.294652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0fa096aa-d79d-425e-a5c3-73899789b571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:38.296294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1141/2281] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:38 2022] DELETE /identity/v3/projects/df8b05dd172d4226af0d7448ff330302 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:38.301812 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.302707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.302991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.322799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:38.324042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:38.353578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:38.383973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1141/2282] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:17:38.391721 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.392003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.392277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.412546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:38.413738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:38.421635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1142/2283] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:38.426446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.436049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857e15-0c11-41d0-80b7-6f03185865d2 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.440587 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.441203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.441478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.441747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.442383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:17:38.442719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.444333 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.455708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1142/2284] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:17:38.459488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.468570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984dd63-5a7e-47fd-a9ae-bbd7f65d1c12 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.473586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.474168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.474441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.474711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.475329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.477233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.503854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1143/2285] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/projects => generated 409 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:38.508028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.517579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09079d0-e274-4218-b347-64f6681f6550 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.571093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.571726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.572003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.572273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.572900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.574822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.595379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1143/2286] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/users => generated 305 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:38.600299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.611706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df092d-d59b-48f3-8ee0-ac1566310baf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.616540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.617161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.617436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.617707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.618345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.619975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.629758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1144/2287] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:38.633674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.643562 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ff3bf-de8f-4823-8f59-98e51f64fc71 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.648768 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110da52-d034-4801-8a59-cd64017ad00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.649370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110da52-d034-4801-8a59-cd64017ad00c admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.649647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110da52-d034-4801-8a59-cd64017ad00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.649920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110da52-d034-4801-8a59-cd64017ad00c admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.650594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110da52-d034-4801-8a59-cd64017ad00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.652213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110da52-d034-4801-8a59-cd64017ad00c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.663358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1144/2288] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:38.667392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.677233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bfa23-f2af-4eb2-b7ac-4edcf91a0a3c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.681783 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.682418 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.682697 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.682969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] PATH_INFO: `/v3/projects/6b1170e2440442c58bf6c04a9a5e7f3f/users/449c19ccacab44649f6012641643e4a6/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.697384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1170e2440442c58bf6c04a9a5e7f3f, user_id=449c19ccacab44649f6012641643e4a6, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.700094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.713980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1145/2289] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:38 2022] PUT /identity/v3/projects/6b1170e2440442c58bf6c04a9a5e7f3f/users/449c19ccacab44649f6012641643e4a6/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:38.725827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.726057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.726331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.753546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] MFA Rules not processed for user `449c19ccacab44649f6012641643e4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:38.754750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:38.817277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1145/2290] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:38.821898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.870931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:38.883186 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbed7e-bc50-4af3-b7a4-8e7370d3bfe4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.888289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.888886 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.889162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.889429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.890063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.891991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:38.918081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1146/2291] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/projects => generated 407 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:38.922205 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:38.932641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160df51-63e1-483f-be37-798ee6da4516 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:38.960674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:38.989865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:38.990394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:38.990926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:38.991199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:38.991855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:38.993761 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.013627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1146/2292] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:38 2022] POST /identity/v3/users => generated 310 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:17:39.017657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:39.027815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb93601-47b2-4794-98f4-eea4d598f47d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:39.032432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:39.033022 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.033303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.033572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.034210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:39.036343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.046734 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1147/2293] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:39.051142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:39.061398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c1328-e5bd-472e-98b1-6b54ad455a53 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:39.066028 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:39.066675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.066955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.067229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] PATH_INFO: `/v3/projects/249fe26505a948079a866f0ae7947da3/users/0f310fe496ae43af8af517801fb7afff/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.082551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] RBAC: Authorizing `identity:create_grant(project_id=249fe26505a948079a866f0ae7947da3, user_id=0f310fe496ae43af8af517801fb7afff, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:39.085251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.099045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1147/2294] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:39 2022] PUT /identity/v3/projects/249fe26505a948079a866f0ae7947da3/users/0f310fe496ae43af8af517801fb7afff/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:39.102861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:39.113567 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1685be-3b63-4463-b628-64bf23cabcac admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:39.173599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df24ef9-2861-49d6-bd5c-ba2dcd52cc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:39.174210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df24ef9-2861-49d6-bd5c-ba2dcd52cc4d admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.174490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df24ef9-2861-49d6-bd5c-ba2dcd52cc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.174770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df24ef9-2861-49d6-bd5c-ba2dcd52cc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.176128 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df24ef9-2861-49d6-bd5c-ba2dcd52cc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:39.177719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df24ef9-2861-49d6-bd5c-ba2dcd52cc4d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.188078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1148/2295] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:39.192295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:39.202278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f687a-0293-45fb-bdbc-744c2025839d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:39.206756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:39.207392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.207695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.207961 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] PATH_INFO: `/v3/projects/249fe26505a948079a866f0ae7947da3/users/0f310fe496ae43af8af517801fb7afff/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.209832 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] RBAC: Authorizing `identity:create_grant(project_id=249fe26505a948079a866f0ae7947da3, user_id=0f310fe496ae43af8af517801fb7afff, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:39.212484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.223206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1148/2296] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:39 2022] PUT /identity/v3/projects/249fe26505a948079a866f0ae7947da3/users/0f310fe496ae43af8af517801fb7afff/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:39.233245 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.233527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.233798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.261527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] MFA Rules not processed for user `0f310fe496ae43af8af517801fb7afff`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:39.262640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:39.333660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1149/2297] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:39 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:39.342302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:39.344218 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:39.401944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23461ca-8a84-4727-82bc-803c3df6966b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:39.405555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:39.406150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.406421 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.406685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.416455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:39.417871 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.427057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1149/2298] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:39 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:39.575789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:39.586164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff2615-ad23-437b-ada7-e0f47373e1a7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:39.589733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa1aea-e16b-43ac-8b23-ce05ec31f080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:39.590324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1aea-e16b-43ac-8b23-ce05ec31f080 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:39.590624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1aea-e16b-43ac-8b23-ce05ec31f080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:39.590895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1aea-e16b-43ac-8b23-ce05ec31f080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:39.634203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:39.647633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa1aea-e16b-43ac-8b23-ce05ec31f080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:39.649083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa1aea-e16b-43ac-8b23-ce05ec31f080 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:39.657785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1150/2299] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:39 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:47.970079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:48.034803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a0662-45d4-4f0c-981a-d101880638aa service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:48.039234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:48.039983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:48.040255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:48.040519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] PATH_INFO: `/v3/users/449c19ccacab44649f6012641643e4a6` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:48.041610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] RBAC: Authorizing `identity:delete_user(user_id=449c19ccacab44649f6012641643e4a6)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:48.043561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:48.071276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:48.089144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449c19ccacab44649f6012641643e4a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.089784 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449c19ccacab44649f6012641643e4a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.090382 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449c19ccacab44649f6012641643e4a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.097633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449c19ccacab44649f6012641643e4a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.105061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449c19ccacab44649f6012641643e4a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.122684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1150/2300] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:47 2022] DELETE /identity/v3/users/449c19ccacab44649f6012641643e4a6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:48.127224 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:48.127705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:48.127808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:48.146283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:48.147487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:48.204626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1151/2301] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:48.214273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:48.215639 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:48.271772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993b8c-7622-4d5e-8997-cb6ef652092a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:48.275347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:48.275976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:48.276254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:48.276527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:48.285232 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:48.286640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:48.295868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1151/2302] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:48.558840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:48.560309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:48.574702 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e25ecd-830f-43b4-8acb-faef41aaebf6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:48.582530 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:48.583260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:48.583683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:48.583856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] PATH_INFO: `/v3/projects/6b1170e2440442c58bf6c04a9a5e7f3f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:48.584967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1170e2440442c58bf6c04a9a5e7f3f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:48.586880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:48.634438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1170e2440442c58bf6c04a9a5e7f3f', 'request_id': 'req-27960f06-eef1-4fba-b2d9-11bcd990617d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.640678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1170e2440442c58bf6c04a9a5e7f3f', 'request_id': 'req-27960f06-eef1-4fba-b2d9-11bcd990617d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.648086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1170e2440442c58bf6c04a9a5e7f3f', 'request_id': 'req-27960f06-eef1-4fba-b2d9-11bcd990617d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.658861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] The token cache is being invalidate because project 6b1170e2440442c58bf6c04a9a5e7f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:48.659799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.660424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.661904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1152/2303] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:48 2022] DELETE /identity/v3/projects/6b1170e2440442c58bf6c04a9a5e7f3f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:48.665831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:48.675831 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dae7aa-c778-42c1-867f-79feda7db2fb service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:48.730975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:48.731632 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:48.731911 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:48.732182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] PATH_INFO: `/v3/users/0f310fe496ae43af8af517801fb7afff` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:48.733316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f310fe496ae43af8af517801fb7afff)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:48.735294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:48.779528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f310fe496ae43af8af517801fb7afff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.780165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f310fe496ae43af8af517801fb7afff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.780759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f310fe496ae43af8af517801fb7afff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.782718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:48.790440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f310fe496ae43af8af517801fb7afff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.797373 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-22375ecf-dd19-4350-838e-35eaf9b5600c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f310fe496ae43af8af517801fb7afff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:48.815155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1152/2304] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:17:48 2022] DELETE /identity/v3/users/0f310fe496ae43af8af517801fb7afff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:49.077328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.079173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:49.097612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27960f06-eef1-4fba-b2d9-11bcd990617d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.153681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.154342 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.154572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.154841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] PATH_INFO: `/v3/projects/249fe26505a948079a866f0ae7947da3` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.155976 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249fe26505a948079a866f0ae7947da3)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.157889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.168482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:49.204091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249fe26505a948079a866f0ae7947da3', 'request_id': 'req-c1f99ddc-6b51-4790-8328-caddb793b4b6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:49.210682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249fe26505a948079a866f0ae7947da3', 'request_id': 'req-c1f99ddc-6b51-4790-8328-caddb793b4b6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:49.217196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249fe26505a948079a866f0ae7947da3', 'request_id': 'req-c1f99ddc-6b51-4790-8328-caddb793b4b6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:49.227794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] The token cache is being invalidate because project 249fe26505a948079a866f0ae7947da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:17:49.228080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:49.228720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1f99ddc-6b51-4790-8328-caddb793b4b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:17:49.230181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1153/2305] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:17:49 2022] DELETE /identity/v3/projects/249fe26505a948079a866f0ae7947da3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:49.236160 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.236627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.236732 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.256351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:49.257478 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:49.314500 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1153/2306] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:49.323616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.323906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.324185 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.344079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:49.345260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:49.353558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1154/2307] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:49.357771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.367705 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728eda57-c93b-487d-9fb6-b321e8a121e1 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.372186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.372777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.373049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.373315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.373952 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:17:49.374250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.375865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.387270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1154/2308] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:17:49.391541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.401011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74c2c3-d8ad-4d03-8b89-d58af7359a20 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.405563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.406157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.406430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.406698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.407322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.409798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.435884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1155/2309] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:49 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:49.440013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.450219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86f57-e135-4f94-bfc0-5ca6c798babc admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.506097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.506750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.506972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.507240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.507901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.509954 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.530172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1155/2310] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:49 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:49.534385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.544554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fed238-fbae-453b-947e-09f702231fd6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.549120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.549736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.550012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.550279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.550912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.552547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.562225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1156/2311] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:49.566440 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.577243 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64430267-5080-4a3a-a64b-c343a03aff97 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.582465 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad7aee-2d6e-4033-9f2e-c8081dc83d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.583059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad7aee-2d6e-4033-9f2e-c8081dc83d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.583335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad7aee-2d6e-4033-9f2e-c8081dc83d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.583628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad7aee-2d6e-4033-9f2e-c8081dc83d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.584262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad7aee-2d6e-4033-9f2e-c8081dc83d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.585826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad7aee-2d6e-4033-9f2e-c8081dc83d63 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.596203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1156/2312] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:49.600488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.610496 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd19634-2331-445c-8e58-f2ca0b99192a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.615706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.616358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.617947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.617947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] PATH_INFO: `/v3/projects/0fa103dda8c34135af4c1a3913ccb564/users/211be7ccf81649db928d8efd0a91426f/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.632467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa103dda8c34135af4c1a3913ccb564, user_id=211be7ccf81649db928d8efd0a91426f, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.635238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.648900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1157/2313] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:49 2022] PUT /identity/v3/projects/0fa103dda8c34135af4c1a3913ccb564/users/211be7ccf81649db928d8efd0a91426f/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:49.656443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e1e91-c792-42ad-8842-867fdf47d7fb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.656736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e1e91-c792-42ad-8842-867fdf47d7fb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.657033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e1e91-c792-42ad-8842-867fdf47d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.675764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ec4e1e91-c792-42ad-8842-867fdf47d7fb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:49.676942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ec4e1e91-c792-42ad-8842-867fdf47d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:49.705755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:49.736239 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1157/2314] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:49.746202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:49.747919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:49.804611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a09c0-c877-4bdf-b70e-198984331c6c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:49.808157 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:49.808754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:49.809029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:49.809298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:49.818674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:49.820127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:49.829352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1158/2315] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:53.724766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:53.725268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:53.725368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:53.760919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] MFA Rules not processed for user `211be7ccf81649db928d8efd0a91426f`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:53.762057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:53.842113 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1158/2316] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:53.847582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:53.859337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23e890-8d7d-4807-ac3f-d949ee9f21f2 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:53.865114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:53.866063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:53.866414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:53.866758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:53.867636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:53.869584 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:53.899170 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1159/2317] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:17:53 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:53.903438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:53.915233 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0760d9-0069-45a1-8e9c-5cfce1ae31cf None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:53.997995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:53.998603 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:53.998878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:53.999147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:53.999823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:54.001716 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:54.041549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1159/2318] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:17:53 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:17:54.043906 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:54.055693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f4a3e-a7e4-4528-8817-7030323f93c6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:54.060491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:54.061109 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:54.061390 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:54.061664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:54.062313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:54.063995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:54.075669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1160/2319] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:54.079923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:54.092464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ef69c-c3c6-4a20-8291-1303a0843c06 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:54.098370 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:54.099133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:54.099519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:54.099801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] PATH_INFO: `/v3/projects/051f15ffb9c44cafa699693c33e2b14a/users/ab74860e0e5346c085bbb4f6e367d2a2/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:54.117687 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=051f15ffb9c44cafa699693c33e2b14a, user_id=ab74860e0e5346c085bbb4f6e367d2a2, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:54.120488 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:54.135408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1160/2320] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:54 2022] PUT /identity/v3/projects/051f15ffb9c44cafa699693c33e2b14a/users/ab74860e0e5346c085bbb4f6e367d2a2/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:17:54.140107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:54.150095 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:54.155456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aba06c-2047-4ac6-88ff-1f7ae4da304e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:54.234041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f0815-dd81-4173-bfc1-6591a0ddfa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:54.234573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f0815-dd81-4173-bfc1-6591a0ddfa16 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:54.234885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f0815-dd81-4173-bfc1-6591a0ddfa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:54.235159 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f0815-dd81-4173-bfc1-6591a0ddfa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:54.235840 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f0815-dd81-4173-bfc1-6591a0ddfa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:54.238028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f0815-dd81-4173-bfc1-6591a0ddfa16 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:54.248755 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1161/2321] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:17:54.253017 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:54.263483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3d26f-668c-4942-8a35-795b5b3d1bec admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:54.268608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:54.269277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:54.269553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:54.269820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] PATH_INFO: `/v3/projects/051f15ffb9c44cafa699693c33e2b14a/users/ab74860e0e5346c085bbb4f6e367d2a2/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:54.271783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051f15ffb9c44cafa699693c33e2b14a, user_id=ab74860e0e5346c085bbb4f6e367d2a2, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:54.274467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:54.286260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1161/2322] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:17:54 2022] PUT /identity/v3/projects/051f15ffb9c44cafa699693c33e2b14a/users/ab74860e0e5346c085bbb4f6e367d2a2/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:17:58.562348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58a30-13de-4605-b8fb-bacc13ec1eb1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:58.563611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58a30-13de-4605-b8fb-bacc13ec1eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:58.564080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58a30-13de-4605-b8fb-bacc13ec1eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:58.593805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-89d58a30-13de-4605-b8fb-bacc13ec1eb1 None None] MFA Rules not processed for user `ab74860e0e5346c085bbb4f6e367d2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:58.594959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-89d58a30-13de-4605-b8fb-bacc13ec1eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:58.668104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1162/2323] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:17:58.685497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:17:58.687269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:58.706851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:17:58.775358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feabfe-a078-4db4-9757-5d152c5f0c16 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:17:58.780878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:17:58.781490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:58.781792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:58.782061 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:58.843211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:17:58.844881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:17:58.855898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1162/2324] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:17:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:17:59.852349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:17:59.852902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:17:59.852902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:17:59.872071 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] MFA Rules not processed for user `211be7ccf81649db928d8efd0a91426f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:17:59.873248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:17:59.880986 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1163/2325] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:17:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:18:08.850951 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=49 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:18:09.023998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:18:09.026641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:09.083993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff352-6fda-4a40-ab12-0945bd026567 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:18:09.087614 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:18:09.088215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:18:09.088493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:18:09.088762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:18:09.090482 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:09.150010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:18:09.151455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:18:09.169129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1163/2326] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:18:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:18:09.866685 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=64 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:18:17.258068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:18:17.267823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b245836-6058-49f4-a33b-206ea1d6ec5f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:18:17.272099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:18:17.272695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:18:17.272972 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:18:17.273242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:18:17.274987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:17.287983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:18:17.289408 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:18:17.300089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1164/2327] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:18:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:18:18.028072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:18:18.029513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:18.088516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a9428-f438-48aa-b7b0-8465b547a108 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:18:18.092033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:18:18.092636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:18:18.092912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:18:18.093179 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:18:18.094942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:18.109359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:18:18.111375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:18:18.123670 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1164/2328] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:18:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:18:20.283568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:18:20.285136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:20.297644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896060-6f58-47c1-bde5-e3b978233e00 service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:18:20.301349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:18:20.301967 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:18:20.302246 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:18:20.302520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:18:20.304546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:18:20.368233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:18:20.370064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e92fa-7efb-43fd-914c-9dddb9a98b7d service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:18:20.380698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1165/2329] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:18:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:18:28.118349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=78 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:18:30.375469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=34 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:18:33.909209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:18:33.918476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc8379-e0d8-49ac-955e-3aa574539ddb service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:18:33.922070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:18:33.922680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:18:33.922956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:18:33.923230 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:18:33.932328 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:18:33.933785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:18:33.942966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1165/2330] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:18:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:18:43.938874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:09.426882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:09.427343 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:09.427550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:09.446397 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] MFA Rules not processed for user `52173fbead6e4b499a99a6979c9a9898`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:09.447619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:09.502687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1166/2331] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:19.496005 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=25 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:21.770155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:21.778929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db6f5-7fdb-4b60-ae1c-f2a94e239e90 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:21.782876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:21.783545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:21.784009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:21.784274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:21.786188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:21.866277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:21.868185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:21.879041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1166/2332] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:22.266678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:22.276720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce11ac6-3475-4c4e-a6a8-aff9c3903a25 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:22.280561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:22.281165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:22.282062 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:22.282633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:22.284392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:22.297341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:22.298854 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:22.309600 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1167/2333] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:26.316153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:26.318254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:26.336490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced9ed8-43a7-4d93-9aa4-db4e20764a3b service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:26.352955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:26.353630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:26.354072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:26.354495 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] PATH_INFO: `/v3/users/2e63c3f83f3d41018e6330f26ed7b8a0` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:26.356004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e63c3f83f3d41018e6330f26ed7b8a0)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:26.359058 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:26.447728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e63c3f83f3d41018e6330f26ed7b8a0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.448847 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e63c3f83f3d41018e6330f26ed7b8a0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.449992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e63c3f83f3d41018e6330f26ed7b8a0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.462258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e63c3f83f3d41018e6330f26ed7b8a0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.475280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-08f14545-0b40-466f-902c-5bece7a15d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e63c3f83f3d41018e6330f26ed7b8a0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.503156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1167/2334] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:26 2022] DELETE /identity/v3/users/2e63c3f83f3d41018e6330f26ed7b8a0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:26.691724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:26.704951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada4cd-3d46-4aee-b12d-f58481bd2369 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:26.801823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:26.802583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:26.802937 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:26.803207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] PATH_INFO: `/v3/projects/5af47e2321de4d299ff3a7fb3d3d9e3c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:26.804399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af47e2321de4d299ff3a7fb3d3d9e3c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:26.806636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:26.860437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af47e2321de4d299ff3a7fb3d3d9e3c', 'request_id': 'req-1efb2061-e473-4a45-891f-598a6eac35d3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.867084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af47e2321de4d299ff3a7fb3d3d9e3c', 'request_id': 'req-1efb2061-e473-4a45-891f-598a6eac35d3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.875193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af47e2321de4d299ff3a7fb3d3d9e3c', 'request_id': 'req-1efb2061-e473-4a45-891f-598a6eac35d3'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.887021 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] The token cache is being invalidate because project 5af47e2321de4d299ff3a7fb3d3d9e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:26.887309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.887983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1efb2061-e473-4a45-891f-598a6eac35d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:26.889476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1168/2335] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:26 2022] DELETE /identity/v3/projects/5af47e2321de4d299ff3a7fb3d3d9e3c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:26.895968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:26.896351 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:26.896713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:26.916900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:26.918119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:26.985490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1168/2336] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:26.993318 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:26.993560 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:26.993815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.014521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:27.015769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:27.024521 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1169/2337] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:27.028880 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.038343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9322d-99e6-4e54-a315-eb2057b456de None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.042849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.043474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.043750 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.044016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.044657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:27.044957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.046537 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.048054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:27.063503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1169/2338] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:27.064554 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.074583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7b1c2-ba33-450e-8a56-9d04489c29e0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.080737 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.081758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.082413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.082413 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.083068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.085632 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.113999 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1170/2339] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:27 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:27.118801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.128920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ca9be-07b3-4b78-98aa-860b80562868 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.185238 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.185840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.186115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.186379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.186996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.188914 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.211944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1170/2340] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:27 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:27.216161 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.226306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365da2f-f3e3-4db3-92c0-03e85c34c2a4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.230901 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.231534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.231815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.232082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.232848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.234494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.245341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1171/2341] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:27.249326 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.259771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016945-5ea5-4152-a6e5-321703645b83 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.264320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a820dea-0c91-44cc-9c6e-0389e33d78ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.265023 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a820dea-0c91-44cc-9c6e-0389e33d78ed admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.265300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a820dea-0c91-44cc-9c6e-0389e33d78ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.265564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a820dea-0c91-44cc-9c6e-0389e33d78ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.266196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a820dea-0c91-44cc-9c6e-0389e33d78ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.267829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a820dea-0c91-44cc-9c6e-0389e33d78ed admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.279051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1171/2342] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:27.283271 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.296516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e0dc8-3d1e-485f-b811-d21319b215ef admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.308114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.308787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.309064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.309336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] PATH_INFO: `/v3/projects/f200b695e287478fa5788633f92fbe35/users/382a2daeb9ee4fc78a3999da4245680d/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.327773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f200b695e287478fa5788633f92fbe35, user_id=382a2daeb9ee4fc78a3999da4245680d, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.330587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.359621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1172/2343] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:27 2022] PUT /identity/v3/projects/f200b695e287478fa5788633f92fbe35/users/382a2daeb9ee4fc78a3999da4245680d/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:27.361966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.363610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.364060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.398592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] MFA Rules not processed for user `382a2daeb9ee4fc78a3999da4245680d`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:27.400033 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:27.442620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.445758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:27.462473 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:27.500201 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1172/2344] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:27.504471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.529376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23ce2-9d8a-4bc9-b8fe-cb74c72444a1 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.534620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.535478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.535761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.536031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] PATH_INFO: `/v3/users/f95c1cd09e1042abbb5725f241ce147d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.537700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95c1cd09e1042abbb5725f241ce147d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.539657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.579111 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbab50-62eb-4dc8-8009-527fad886465 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.585131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.586272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.586611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.587000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.587787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.589971 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.603433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:27.605293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95c1cd09e1042abbb5725f241ce147d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:27.606349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95c1cd09e1042abbb5725f241ce147d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:27.618462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95c1cd09e1042abbb5725f241ce147d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:27.621373 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95c1cd09e1042abbb5725f241ce147d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:27.634875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95c1cd09e1042abbb5725f241ce147d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:27.640910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1173/2345] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:27 2022] POST /identity/v3/projects => generated 361 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:27.642742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.645073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:27.667803 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1173/2346] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:27 2022] DELETE /identity/v3/users/f95c1cd09e1042abbb5725f241ce147d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:27.738095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c427d-6c6e-4494-ad35-9da05e13dac8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.841957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.843727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.843727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.843727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.844026 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.846430 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.879194 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1174/2347] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:27 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:27.880713 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.892207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bacf5b8-d23b-4769-8198-7450c69a091f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.898100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.898719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.899000 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.899525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.899955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.901572 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.910547 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.913077 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:27.930458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1174/2348] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:27.933902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e87c-1d13-4f26-a7da-927e4e7ad9d7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.934993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:27.939198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.940641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.950150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.950594 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] PATH_INFO: `/v3/projects/ec4a4c53c1c744688b3bf91e806286bf` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.952385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c602af6-48cd-4d05-8b30-a313f2cfd7eb admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:27.952700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4a4c53c1c744688b3bf91e806286bf)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.954818 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:27.959630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:27.960422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:27.960807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:27.961178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] PATH_INFO: `/v3/projects/11035c7e2616474ba49071a0e023710b/users/9c111948b4fe4581bf8b19b06ab94460/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:27.980717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11035c7e2616474ba49071a0e023710b, user_id=9c111948b4fe4581bf8b19b06ab94460, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:27.985290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.015759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1175/2349] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:27 2022] PUT /identity/v3/projects/11035c7e2616474ba49071a0e023710b/users/9c111948b4fe4581bf8b19b06ab94460/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:28.019488 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.037821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4a4c53c1c744688b3bf91e806286bf', 'request_id': 'req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:28.048367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4a4c53c1c744688b3bf91e806286bf', 'request_id': 'req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:28.058741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4a4c53c1c744688b3bf91e806286bf', 'request_id': 'req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:28.072149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] The token cache is being invalidate because project ec4a4c53c1c744688b3bf91e806286bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:28.072518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:28.073302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a9d649f2-f5d0-40cd-9157-1d6887cfcf2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:28.075231 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1175/2350] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:27 2022] DELETE /identity/v3/projects/ec4a4c53c1c744688b3bf91e806286bf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:28.081591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255dc8-2d81-4c91-819c-89f073be4655 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.081968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255dc8-2d81-4c91-819c-89f073be4655 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.082343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255dc8-2d81-4c91-819c-89f073be4655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.101874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:28.109821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-4d255dc8-2d81-4c91-819c-89f073be4655 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:28.114156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4d255dc8-2d81-4c91-819c-89f073be4655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.135499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:28.140036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccead501-60f1-4813-8ff9-1360c2e7354e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.210770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1176/2351] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:28.212331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.212621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.212891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.217383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.217998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.218280 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.218552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.219200 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.220860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.231819 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1176/2352] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:28.235933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.237872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.243004 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:28.244414 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.251207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42534156-286a-40e6-af44-5683276f0922 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.255968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1177/2353] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:28.259714 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.260973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042749ca-75fd-4542-b3eb-c2f00bdf79c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.261748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042749ca-75fd-4542-b3eb-c2f00bdf79c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.262126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042749ca-75fd-4542-b3eb-c2f00bdf79c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.262648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042749ca-75fd-4542-b3eb-c2f00bdf79c6 admin admin] PATH_INFO: `/v3/projects/11035c7e2616474ba49071a0e023710b/users/9c111948b4fe4581bf8b19b06ab94460/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.264843 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042749ca-75fd-4542-b3eb-c2f00bdf79c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11035c7e2616474ba49071a0e023710b, user_id=9c111948b4fe4581bf8b19b06ab94460, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.268141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042749ca-75fd-4542-b3eb-c2f00bdf79c6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.272607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3eadf-ad3a-469b-9a28-4b78d44a57d5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.278310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.278920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.279195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.279514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.280169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:28.280468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.282046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.291531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1177/2354] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:28 2022] PUT /identity/v3/projects/11035c7e2616474ba49071a0e023710b/users/9c111948b4fe4581bf8b19b06ab94460/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:28.299945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1178/2355] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 07:19:28.300894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.301409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.301696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.301996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.318697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bccf17-e862-4620-858d-aee5fcea6ab1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.340694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] MFA Rules not processed for user `9c111948b4fe4581bf8b19b06ab94460`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:28.342292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.397640 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.398240 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.398499 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.398770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.399406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.401514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.432129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1179/2356] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:28 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:28.436387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.443565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1178/2357] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:28.452275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.454337 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.513701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d8738-2281-44fb-9970-d2a3df3aa899 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.519063 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.519655 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026d844-8a6f-4087-80ff-c59832fbb159 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.519941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.520443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.521375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.522292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.524301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.549585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:28.556120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1180/2358] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:28 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:28.557356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.563166 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:28.569457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8866e-620b-4fe1-9aa8-6964619ec0f2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.576996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.577759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.578047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.578327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.578982 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.580634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.581030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953dda0-60aa-4f54-a06a-11d99fe3e575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.581711 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953dda0-60aa-4f54-a06a-11d99fe3e575 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.582053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953dda0-60aa-4f54-a06a-11d99fe3e575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.582392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953dda0-60aa-4f54-a06a-11d99fe3e575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.592012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1181/2359] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:28.595851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.606045 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a34d46-91f6-47f2-980c-c5d668c591ec admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.610865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.611501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.611786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.612062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.612972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.615087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.627718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1182/2360] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:28.631923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.642403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a731d9-d9c3-4d8a-9371-2d19111dbc9a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.647672 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.648723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.648723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.648925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] PATH_INFO: `/v3/projects/2445c31c35bb442493befe50f68724bf/users/b1a04fc5bdf6407580c598b494a68d6f/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.651515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953dda0-60aa-4f54-a06a-11d99fe3e575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.653821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953dda0-60aa-4f54-a06a-11d99fe3e575 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.664666 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2445c31c35bb442493befe50f68724bf, user_id=b1a04fc5bdf6407580c598b494a68d6f, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.666388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1179/2361] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:28.668783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.691706 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1183/2362] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:28 2022] PUT /identity/v3/projects/2445c31c35bb442493befe50f68724bf/users/b1a04fc5bdf6407580c598b494a68d6f/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:28.704599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.704599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.704599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.734293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] MFA Rules not processed for user `b1a04fc5bdf6407580c598b494a68d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:28.735611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.751813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.815099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1583-bb3d-4ee2-bea3-e4cc3faf22fa admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.819208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.819449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1180/2363] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:28.823801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.824087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.824368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.826225 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.831201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:28.832853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:28.893025 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.894490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.906713 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1184/2364] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:28.924791 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d61eba-2211-412f-9853-768d366fa53f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:28.930933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a903f7-41e1-484b-bb18-6245b7f2f886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:28.931571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a903f7-41e1-484b-bb18-6245b7f2f886 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:28.931938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a903f7-41e1-484b-bb18-6245b7f2f886 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:28.932187 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a903f7-41e1-484b-bb18-6245b7f2f886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:28.937295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=100 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:28.941361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a903f7-41e1-484b-bb18-6245b7f2f886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:28.944611 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a903f7-41e1-484b-bb18-6245b7f2f886 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:28.956583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1181/2365] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:31.625649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:31.686501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cea188-eefc-4d91-8af1-0f0eeac10543 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:31.691008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:31.691645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:31.691918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:31.692188 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] PATH_INFO: `/v3/users/382a2daeb9ee4fc78a3999da4245680d` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:31.693932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382a2daeb9ee4fc78a3999da4245680d)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:31.695875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:31.709617 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:31.749029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382a2daeb9ee4fc78a3999da4245680d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:31.749680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382a2daeb9ee4fc78a3999da4245680d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:31.750278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382a2daeb9ee4fc78a3999da4245680d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:31.757674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382a2daeb9ee4fc78a3999da4245680d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:31.765122 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382a2daeb9ee4fc78a3999da4245680d'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:31.782443 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1185/2366] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:31 2022] DELETE /identity/v3/users/382a2daeb9ee4fc78a3999da4245680d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:31.786955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:31.787463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:31.787559 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:31.806565 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:31.807872 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:31.866179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1182/2367] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:31.875891 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:31.877369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:31.932137 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aef42-335e-4007-bf20-aade370322d4 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:31.935802 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:31.936395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:31.936730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:31.936940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:31.946208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:31.947651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:31.956844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1186/2368] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:32.222522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:32.224507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:32.236432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554bd59-c087-4a58-b81b-40339db37ac0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:32.242319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:32.242947 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:32.243226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:32.243539 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] PATH_INFO: `/v3/projects/f200b695e287478fa5788633f92fbe35` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:32.244682 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] RBAC: Authorizing `identity:delete_project(project_id=f200b695e287478fa5788633f92fbe35)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:32.247291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:32.292581 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f200b695e287478fa5788633f92fbe35', 'request_id': 'req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.298931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f200b695e287478fa5788633f92fbe35', 'request_id': 'req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.306943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f200b695e287478fa5788633f92fbe35', 'request_id': 'req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.317516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] The token cache is being invalidate because project f200b695e287478fa5788633f92fbe35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:32.317853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.318481 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.320009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1183/2369] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:32 2022] DELETE /identity/v3/projects/f200b695e287478fa5788633f92fbe35 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:32.323947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:32.333800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641c04-1c8c-4e9b-96fd-487362399afc service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:32.389577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:32.390199 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:32.390471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:32.390738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] PATH_INFO: `/v3/users/9c111948b4fe4581bf8b19b06ab94460` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:32.391919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c111948b4fe4581bf8b19b06ab94460)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:32.393820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:32.432051 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:32.442917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c111948b4fe4581bf8b19b06ab94460'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.443621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c111948b4fe4581bf8b19b06ab94460'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.444213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c111948b4fe4581bf8b19b06ab94460'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.450944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c111948b4fe4581bf8b19b06ab94460'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.457810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-b767f1e2-fa10-4d3a-a2a8-5c3c51655e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c111948b4fe4581bf8b19b06ab94460'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.475534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1187/2370] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:32 2022] DELETE /identity/v3/users/9c111948b4fe4581bf8b19b06ab94460 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:32.743118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:32.744805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:32.758149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac339b2b-89f2-41cd-bb4f-fabb7d81f2db admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:32.816813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:32.817996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:32.818203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:32.819144 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] PATH_INFO: `/v3/projects/11035c7e2616474ba49071a0e023710b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:32.820301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11035c7e2616474ba49071a0e023710b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:32.822204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:32.843954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:32.869983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11035c7e2616474ba49071a0e023710b', 'request_id': 'req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.876192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11035c7e2616474ba49071a0e023710b', 'request_id': 'req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.883771 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11035c7e2616474ba49071a0e023710b', 'request_id': 'req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.893862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] The token cache is being invalidate because project 11035c7e2616474ba49071a0e023710b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:32.894149 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.894773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-92dbb707-ad1d-43ad-9bb2-c34eb9da30d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:32.896809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1184/2371] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:32 2022] DELETE /identity/v3/projects/11035c7e2616474ba49071a0e023710b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:32.901637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:32.902335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:32.902335 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:32.921555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:32.922755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:32.980918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1188/2372] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:32.989950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ed9c-671b-43d8-a115-621565646566 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:32.990176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ed9c-671b-43d8-a115-621565646566 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:32.990451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ed9c-671b-43d8-a115-621565646566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.010092 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7001ed9c-671b-43d8-a115-621565646566 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:33.011282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7001ed9c-671b-43d8-a115-621565646566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:33.019443 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1185/2373] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:33.024075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.033509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec03da-2a12-4c96-ad84-778e1b24711c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.038037 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.038628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.038908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.039180 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.039867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:33.040168 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.041747 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.052984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1189/2374] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 07:19:33.056994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001ed9c-671b-43d8-a115-621565646566 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.066036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001ed9c-671b-43d8-a115-621565646566 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.071265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.071888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.072169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.072441 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.073078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.074991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.101666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1186/2375] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:33 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:33.105769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.116693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c775bc-f032-4161-9cd9-b75b0592e79a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.170827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.171452 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.171738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.172011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.172644 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.174570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.196267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1190/2376] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:33 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:33.200405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.210617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee1ac-6446-46a7-a720-47225bf6cba7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.215277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.215902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.216178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.216450 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.217086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.218679 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.228429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1187/2377] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:33.232262 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.242700 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda827e-652b-49fe-9271-7723b7f29a78 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.247126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a786a-623b-4dfe-ab86-3f93f5dc3b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.247856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a786a-623b-4dfe-ab86-3f93f5dc3b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.248133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a786a-623b-4dfe-ab86-3f93f5dc3b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.248399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a786a-623b-4dfe-ab86-3f93f5dc3b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.249060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a786a-623b-4dfe-ab86-3f93f5dc3b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.250612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a786a-623b-4dfe-ab86-3f93f5dc3b82 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.261417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1191/2378] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:33.265321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.274973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e3536-7943-4d57-abfa-37b999eb31b2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.279499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.280133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.280414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.280689 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] PATH_INFO: `/v3/projects/c8cd2c66a4fd4a0aad133d477b7603a8/users/655a893fe38249fdb6c51a1e1114242c/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.294889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cd2c66a4fd4a0aad133d477b7603a8, user_id=655a893fe38249fdb6c51a1e1114242c, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.297585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.310717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1188/2379] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:33 2022] PUT /identity/v3/projects/c8cd2c66a4fd4a0aad133d477b7603a8/users/655a893fe38249fdb6c51a1e1114242c/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:33.321460 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.321776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.322050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.348899 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] MFA Rules not processed for user `655a893fe38249fdb6c51a1e1114242c`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:33.350115 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:33.354973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:33.414695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1192/2380] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:33.419133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.480524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ebcc-dc82-420b-b773-8b123272c444 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.485106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.485700 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.485974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.486240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.486863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.489053 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.514677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1189/2381] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:33 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:33.518708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.529214 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4868ba8-cf7e-4029-9569-7ac51bf16efa None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.584075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.584704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.584983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.585257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.585892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.587835 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.609648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1193/2382] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:33 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:33.613766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.624780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37a584-ee83-4fb5-b3ae-1d7c075d4a27 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.629376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.629969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.630247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.630517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.631158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.632979 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.643545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1190/2383] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:33.647618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.657330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0a018-7f5f-4679-8aac-779c442853bb admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.661926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.663056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.663334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.663635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] PATH_INFO: `/v3/projects/a8d26c7f4936424a9c463798454dc9ce/users/118c49f2d1794b65a08c69ab5fb7e37d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.677960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d26c7f4936424a9c463798454dc9ce, user_id=118c49f2d1794b65a08c69ab5fb7e37d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.680656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.693733 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1194/2384] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:33 2022] PUT /identity/v3/projects/a8d26c7f4936424a9c463798454dc9ce/users/118c49f2d1794b65a08c69ab5fb7e37d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:33.698597 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.708472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16be8d0-7a9a-45ab-9bdd-8bf4f9a358d0 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.721165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:33.769215 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535e6a-d423-408e-b9de-149dd4ea55a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.769768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535e6a-d423-408e-b9de-149dd4ea55a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.770049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535e6a-d423-408e-b9de-149dd4ea55a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.770321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535e6a-d423-408e-b9de-149dd4ea55a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.770983 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535e6a-d423-408e-b9de-149dd4ea55a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.773180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535e6a-d423-408e-b9de-149dd4ea55a3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.783133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1191/2385] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:33.787507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.798855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db931c-a87a-40ee-b72b-7be944dd153d admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:33.804287 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:33.804942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.805219 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.805489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] PATH_INFO: `/v3/projects/a8d26c7f4936424a9c463798454dc9ce/users/118c49f2d1794b65a08c69ab5fb7e37d/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.807437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d26c7f4936424a9c463798454dc9ce, user_id=118c49f2d1794b65a08c69ab5fb7e37d, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:33.810069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:33.830652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1195/2386] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:33 2022] PUT /identity/v3/projects/a8d26c7f4936424a9c463798454dc9ce/users/118c49f2d1794b65a08c69ab5fb7e37d/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:33.832372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:33.832829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:33.832985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:33.863896 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] MFA Rules not processed for user `118c49f2d1794b65a08c69ab5fb7e37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:33.865621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:33.943139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1192/2387] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:33.954069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:33.955829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:34.014384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45acea7-d4b4-49d5-a23a-ffafa6de940e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:34.018224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeb926-23ce-4b5d-9c73-2782df310857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:34.018819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeb926-23ce-4b5d-9c73-2782df310857 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:34.019096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeb926-23ce-4b5d-9c73-2782df310857 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:34.019502 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeb926-23ce-4b5d-9c73-2782df310857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:34.028586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeb926-23ce-4b5d-9c73-2782df310857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:34.030099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeb926-23ce-4b5d-9c73-2782df310857 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:34.039546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1196/2388] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:34.178402 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:34.237938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78ea6b-0e82-4976-b29c-af8be92c04cb None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:34.242961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:34.243610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:34.243888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:34.244156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] PATH_INFO: `/v3/users/655a893fe38249fdb6c51a1e1114242c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:34.245256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655a893fe38249fdb6c51a1e1114242c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:34.247177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:34.285944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:34.291578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655a893fe38249fdb6c51a1e1114242c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.292469 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655a893fe38249fdb6c51a1e1114242c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.299542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655a893fe38249fdb6c51a1e1114242c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.300166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655a893fe38249fdb6c51a1e1114242c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.307868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655a893fe38249fdb6c51a1e1114242c'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.325069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1193/2389] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:34 2022] DELETE /identity/v3/users/655a893fe38249fdb6c51a1e1114242c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:34.329501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:34.329783 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:34.330066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:34.336994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:34.350455 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:34.351676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:34.408183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1197/2390] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:34.418078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:34.419507 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:34.476759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011d2d3-5ef8-478c-a907-08fc2e351e45 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:34.480536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:34.481139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:34.481420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:34.481696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:34.491503 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:34.492928 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:34.502313 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1194/2391] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:34.759986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:34.761424 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:34.773924 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bebc3-49ae-4417-8fea-87b00c337a22 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:34.778708 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:34.779344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:34.779660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:34.779955 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] PATH_INFO: `/v3/projects/c8cd2c66a4fd4a0aad133d477b7603a8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:34.781068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cd2c66a4fd4a0aad133d477b7603a8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:34.783109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:34.830094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cd2c66a4fd4a0aad133d477b7603a8', 'request_id': 'req-ffd3dd25-5289-4874-83ae-773f96b8bd4b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.837936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cd2c66a4fd4a0aad133d477b7603a8', 'request_id': 'req-ffd3dd25-5289-4874-83ae-773f96b8bd4b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.844648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cd2c66a4fd4a0aad133d477b7603a8', 'request_id': 'req-ffd3dd25-5289-4874-83ae-773f96b8bd4b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.855193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] The token cache is being invalidate because project c8cd2c66a4fd4a0aad133d477b7603a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:34.855528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.856138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.857650 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1198/2392] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:34 2022] DELETE /identity/v3/projects/c8cd2c66a4fd4a0aad133d477b7603a8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:34.861573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:34.871578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26014e53-5ac9-4dfa-9b9d-b76c419a03cb service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:34.926850 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:34.927498 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:34.927779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:34.928052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] PATH_INFO: `/v3/users/118c49f2d1794b65a08c69ab5fb7e37d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:34.929165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] RBAC: Authorizing `identity:delete_user(user_id=118c49f2d1794b65a08c69ab5fb7e37d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:34.931335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:34.981652 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118c49f2d1794b65a08c69ab5fb7e37d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.982307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118c49f2d1794b65a08c69ab5fb7e37d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.995552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118c49f2d1794b65a08c69ab5fb7e37d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:34.996282 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118c49f2d1794b65a08c69ab5fb7e37d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.004816 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-40332ecc-fa25-42e1-a662-ea8a07b2537e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118c49f2d1794b65a08c69ab5fb7e37d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.006550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:35.026811 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1195/2393] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:34 2022] DELETE /identity/v3/users/118c49f2d1794b65a08c69ab5fb7e37d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:35.306957 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.308433 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:35.324787 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3dd25-5289-4874-83ae-773f96b8bd4b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.385748 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.386413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.386695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.386977 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] PATH_INFO: `/v3/projects/a8d26c7f4936424a9c463798454dc9ce` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.388167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d26c7f4936424a9c463798454dc9ce)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.390097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.422295 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:35.438279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d26c7f4936424a9c463798454dc9ce', 'request_id': 'req-7449ec94-163a-40fc-bdbc-132897ad9fd0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.445515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d26c7f4936424a9c463798454dc9ce', 'request_id': 'req-7449ec94-163a-40fc-bdbc-132897ad9fd0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.451987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d26c7f4936424a9c463798454dc9ce', 'request_id': 'req-7449ec94-163a-40fc-bdbc-132897ad9fd0'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.462620 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] The token cache is being invalidate because project a8d26c7f4936424a9c463798454dc9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:35.462909 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.463576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-7449ec94-163a-40fc-bdbc-132897ad9fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:35.465097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1199/2394] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:35 2022] DELETE /identity/v3/projects/a8d26c7f4936424a9c463798454dc9ce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:35.469938 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.470308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.470582 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.490494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:35.491707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:35.548649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1196/2395] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:35.557413 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.557694 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.557960 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.576991 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:35.578164 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:35.586364 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1200/2396] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:35.590476 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.601031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28553-2430-41db-a2f5-a24cfd7a086c None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.605555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.606154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.606429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.606703 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.607351 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:35.607698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.609302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.621335 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1197/2397] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:35.625475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.634841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e77f5-f5b3-4988-b507-19a14dc6f3d2 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.639374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.639995 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.640278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.640550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.641216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.643156 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.670719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1201/2398] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:35 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:35.674586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.684724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbbf36-eefe-4d79-94f4-757f70669659 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.741493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.742094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.742394 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.742648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.743286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.745270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.765922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1198/2399] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:35 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:35.770006 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.781540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdbb71-c363-4445-9360-8e43fe7f2001 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.786282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.787402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.787722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.787998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.788668 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.790257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.800096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1202/2400] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:35.804553 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.814695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f7b4f-a833-4f24-a767-d778dcc0506d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.819518 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53d908-d61f-4cc8-8161-e8828ee9d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.820110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d908-d61f-4cc8-8161-e8828ee9d559 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.820385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d908-d61f-4cc8-8161-e8828ee9d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.820656 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d908-d61f-4cc8-8161-e8828ee9d559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.821292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53d908-d61f-4cc8-8161-e8828ee9d559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.822884 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53d908-d61f-4cc8-8161-e8828ee9d559 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.833724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1199/2401] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:35.837949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:35.847864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2ecc3-6676-4299-a056-53eb4da87793 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:35.852947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:35.853588 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.853869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.854145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] PATH_INFO: `/v3/projects/3caac8dc98444437864f9b0b413cdf8f/users/c69d2c5326e048538853b6e2a241578f/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.869177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3caac8dc98444437864f9b0b413cdf8f, user_id=c69d2c5326e048538853b6e2a241578f, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:35.871932 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:35.884792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1203/2402] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:35 2022] PUT /identity/v3/projects/3caac8dc98444437864f9b0b413cdf8f/users/c69d2c5326e048538853b6e2a241578f/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:35.894904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:35.895355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:35.895465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:35.923084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] MFA Rules not processed for user `c69d2c5326e048538853b6e2a241578f`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:35.924666 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:35.958920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:35.991086 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1200/2403] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:35.995573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.056346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493e867-ef58-4e3b-811e-df9fbe480e2e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.060935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.061529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.061810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.062081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.062718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.064647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.091220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1204/2404] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:35 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:36.095221 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.106311 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85a5de-4dae-4567-8ce2-135fb23a0ee6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.164912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.165524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.165802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.166074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.168322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.171505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.195444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1201/2405] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:36 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:36.196568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.206868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4a51e-1220-41ed-9aa4-5457753599d1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.211999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.212591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.212866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.213131 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.213766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.217345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.230630 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1205/2406] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:36.232028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.242706 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ca6bf-ec49-40c2-9505-1d6c30168b9c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.247449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.248107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.248387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.248662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] PATH_INFO: `/v3/projects/c07dbe04087f431daef06d5f153e9355/users/fc3a5d55c44c4397861383f3c58ba547/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.261645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.261645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.267922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07dbe04087f431daef06d5f153e9355, user_id=fc3a5d55c44c4397861383f3c58ba547, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.270645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.274073 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52f526-099c-4593-840c-debdcb211dae admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.278798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.279484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.279763 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.280036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] PATH_INFO: `/v3/users/b1a04fc5bdf6407580c598b494a68d6f` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.281155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a04fc5bdf6407580c598b494a68d6f)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.283044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.284761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1202/2407] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:36 2022] PUT /identity/v3/projects/c07dbe04087f431daef06d5f153e9355/users/fc3a5d55c44c4397861383f3c58ba547/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:36.288863 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.307027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:36.339184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a04fc5bdf6407580c598b494a68d6f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:36.340176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a04fc5bdf6407580c598b494a68d6f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:36.340804 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a04fc5bdf6407580c598b494a68d6f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:36.347697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a04fc5bdf6407580c598b494a68d6f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:36.356220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-38ec022d-13c5-4004-97fc-b49212c62880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a04fc5bdf6407580c598b494a68d6f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:36.358004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b9ee9-b51f-4e55-9f62-d36740bed9ad admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.362900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.363538 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.363817 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.364093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.364735 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.366859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.374681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1206/2408] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:36 2022] DELETE /identity/v3/users/b1a04fc5bdf6407580c598b494a68d6f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:36.380550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1203/2409] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:36.381228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.381228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.381228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.382075 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.383952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.396326 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbef06-80b4-4bdf-9ab0-7448254d48ae admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.400406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:36.401584 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.431839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:36.456036 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5def1-04fb-4761-bdbf-312f0398ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.456709 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5def1-04fb-4761-bdbf-312f0398ca0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.456964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5def1-04fb-4761-bdbf-312f0398ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.457247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5def1-04fb-4761-bdbf-312f0398ca0c admin admin] PATH_INFO: `/v3/projects/c07dbe04087f431daef06d5f153e9355/users/fc3a5d55c44c4397861383f3c58ba547/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.459160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5def1-04fb-4761-bdbf-312f0398ca0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07dbe04087f431daef06d5f153e9355, user_id=fc3a5d55c44c4397861383f3c58ba547, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.461953 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5def1-04fb-4761-bdbf-312f0398ca0c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.462935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1207/2410] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:36.472575 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.473139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1204/2411] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:36 2022] PUT /identity/v3/projects/c07dbe04087f431daef06d5f153e9355/users/fc3a5d55c44c4397861383f3c58ba547/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:36.476259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.484551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.484834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.485105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.513088 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] MFA Rules not processed for user `fc3a5d55c44c4397861383f3c58ba547`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:36.514306 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.533908 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d5b55-57e7-4ddf-9d3e-4f065afcce74 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.582412 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.582924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1205/2412] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:36.586306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.586615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.586918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.593490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.594861 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.595553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:36.650341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.651867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.653238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d992265-0931-478a-99e1-e47f465f4fe5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.656870 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.657467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.657740 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.658007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.659760 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.661331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1208/2413] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:36.723923 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.725898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5925-d370-4335-b21f-5e1573e76597 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:36.735049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1206/2414] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:36.941468 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:36.943042 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:36.960503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354bb5a-1140-4f17-9fb5-8c33c4d31530 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:36.967900 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:36.968532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:36.968894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:36.969159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] PATH_INFO: `/v3/projects/2445c31c35bb442493befe50f68724bf` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:36.976516 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2445c31c35bb442493befe50f68724bf)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:36.978983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.032719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2445c31c35bb442493befe50f68724bf', 'request_id': 'req-98e814bb-46be-4444-a014-f71ac026f77e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:37.041771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2445c31c35bb442493befe50f68724bf', 'request_id': 'req-98e814bb-46be-4444-a014-f71ac026f77e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:37.048528 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2445c31c35bb442493befe50f68724bf', 'request_id': 'req-98e814bb-46be-4444-a014-f71ac026f77e'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:37.059560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] The token cache is being invalidate because project 2445c31c35bb442493befe50f68724bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:37.059851 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:37.060497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-98e814bb-46be-4444-a014-f71ac026f77e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:37.061963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1209/2415] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:36 2022] DELETE /identity/v3/projects/2445c31c35bb442493befe50f68724bf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:37.067099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.067939 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.068364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.090252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:37.091070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:37.098302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:37.160609 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1207/2416] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:37.170138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.170361 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.170636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.190110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:37.191327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:37.199669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1210/2417] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:37.203823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.213916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611088b5-4079-4de6-98e2-5047baf6dfd4 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.218492 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.219083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.219360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.219665 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.220309 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:37.220610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.222212 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.234105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1208/2418] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:37.238701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.247856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f31987-45e8-4b72-b20f-9f97495976f5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.252481 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.253069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.253343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.253610 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.254234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.256702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.283368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1211/2419] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:37.287485 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.297201 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00c314-5ee1-44e9-bed8-5f6131c0eb4a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.352206 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.352799 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.353078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.353350 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.353971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.355915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.377459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1209/2420] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:37.382079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.392142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72681800-2a2c-421e-a8af-69d11c7d6831 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.397228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.397814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.398091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.398354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.398988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.400583 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.413866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1212/2421] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:37.414951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.425452 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62076c91-9720-498a-bee3-271ada37daef admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.430056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259661-bb19-417b-8629-aa266da53165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.430645 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259661-bb19-417b-8629-aa266da53165 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.430920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259661-bb19-417b-8629-aa266da53165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.431211 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259661-bb19-417b-8629-aa266da53165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.431934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b259661-bb19-417b-8629-aa266da53165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.433551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b259661-bb19-417b-8629-aa266da53165 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.445512 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1210/2422] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:37.449543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.459965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb754f2-ba31-4334-ba73-dae4a3ec1c55 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.464519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.465153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.465432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.465702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] PATH_INFO: `/v3/projects/2c2906d5294c44d28b6635bc42a43d4b/users/14d9e0b2ee91471b9551d8de302070ff/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.479941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2906d5294c44d28b6635bc42a43d4b, user_id=14d9e0b2ee91471b9551d8de302070ff, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.482570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.495196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1213/2423] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:37 2022] PUT /identity/v3/projects/2c2906d5294c44d28b6635bc42a43d4b/users/14d9e0b2ee91471b9551d8de302070ff/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:37.505058 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.505340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.506162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.533505 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] MFA Rules not processed for user `14d9e0b2ee91471b9551d8de302070ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:37.534725 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:37.596857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1211/2424] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:37.601224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.628402 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:37.663399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64dd7d-b705-41bd-bc7e-a47ed30cf5ae admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.667958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.668549 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.668826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.669096 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.669730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.671638 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.697013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1214/2425] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:37.701004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.712009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a5055-6d60-4076-ba48-04155cd92818 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.715680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:37.770961 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.772129 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.772409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.772675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.773300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.775205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.795796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1212/2426] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:37 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:37.799792 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.811585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f77611-8ea6-472b-9365-e2538dcdb06b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.816200 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.816820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.817101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.817371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.818070 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.819836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.830375 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1215/2427] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:37.834991 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.844859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e3f250-4d55-498d-bd01-948c6427cc20 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.849998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.850640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.850915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.851188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] PATH_INFO: `/v3/projects/cbd76cc801e043cdb3591b9d99dcafda/users/07ff7fb4aee64f9b91e7188c6d1251c3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.866081 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd76cc801e043cdb3591b9d99dcafda, user_id=07ff7fb4aee64f9b91e7188c6d1251c3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.868800 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.881692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1213/2428] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:37 2022] PUT /identity/v3/projects/cbd76cc801e043cdb3591b9d99dcafda/users/07ff7fb4aee64f9b91e7188c6d1251c3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:37.885602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.895282 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0048-dab8-44f3-aa29-a01cd5d3d3fb admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.951660 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d1532-2897-4c25-a2ee-a229c0395feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.952189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1532-2897-4c25-a2ee-a229c0395feb admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.952461 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1532-2897-4c25-a2ee-a229c0395feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.952726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1532-2897-4c25-a2ee-a229c0395feb admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.953359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d1532-2897-4c25-a2ee-a229c0395feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.954922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d1532-2897-4c25-a2ee-a229c0395feb admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:37.964925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1216/2429] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:37.969099 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:37.980324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd87bb-a61b-4bdd-b5b7-2c4c7b71c003 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:37.985674 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:37.986327 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:37.986605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:37.986877 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] PATH_INFO: `/v3/projects/cbd76cc801e043cdb3591b9d99dcafda/users/07ff7fb4aee64f9b91e7188c6d1251c3/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:37.989084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd76cc801e043cdb3591b9d99dcafda, user_id=07ff7fb4aee64f9b91e7188c6d1251c3, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:37.991789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:38.002699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1214/2430] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:37 2022] PUT /identity/v3/projects/cbd76cc801e043cdb3591b9d99dcafda/users/07ff7fb4aee64f9b91e7188c6d1251c3/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:38.012785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:38.013071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:38.013346 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:38.043133 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] MFA Rules not processed for user `07ff7fb4aee64f9b91e7188c6d1251c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:38.044399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:38.111099 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1217/2431] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:38.123764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:38.125622 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:38.185029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6baf9-1e7a-4451-b180-3237c1b6519c admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:38.188714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10b1aa-5fa5-4e9a-bd11-2ee177d8308d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:38.189315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10b1aa-5fa5-4e9a-bd11-2ee177d8308d service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:38.189589 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10b1aa-5fa5-4e9a-bd11-2ee177d8308d service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:38.189855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10b1aa-5fa5-4e9a-bd11-2ee177d8308d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:38.199407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10b1aa-5fa5-4e9a-bd11-2ee177d8308d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:38.200848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10b1aa-5fa5-4e9a-bd11-2ee177d8308d service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:38.210675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1215/2432] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:38.358860 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:38.383352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:38.433561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712634ce-4b5e-4938-80e0-34b999fae836 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:38.444715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:38.445561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:38.445986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:38.446400 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] PATH_INFO: `/v3/users/14d9e0b2ee91471b9551d8de302070ff` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:38.447838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d9e0b2ee91471b9551d8de302070ff)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:38.452237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:38.507102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d9e0b2ee91471b9551d8de302070ff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:38.509054 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d9e0b2ee91471b9551d8de302070ff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:38.509729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d9e0b2ee91471b9551d8de302070ff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:38.517752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d9e0b2ee91471b9551d8de302070ff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:38.526636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d9e0b2ee91471b9551d8de302070ff'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:38.543803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1218/2433] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:38 2022] DELETE /identity/v3/users/14d9e0b2ee91471b9551d8de302070ff => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:38.548480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:38.548762 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:38.549030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:38.570133 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:38.571319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:38.627908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1216/2434] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:38.638167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:38.639611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:38.694959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ddb7c-8c2d-4333-95e2-ba80224116d9 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:38.698637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:38.699235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:38.699563 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:38.699838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:38.708801 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:38.710149 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:38.719371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1219/2435] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:38.988163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:38.989563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:39.000541 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:39.007733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4a61d-ac83-4948-8ea2-264344302a64 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:39.013650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:39.014339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.015140 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.015446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] PATH_INFO: `/v3/projects/2c2906d5294c44d28b6635bc42a43d4b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.016570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2906d5294c44d28b6635bc42a43d4b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:39.018480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:39.064197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2906d5294c44d28b6635bc42a43d4b', 'request_id': 'req-88f8432a-617f-4079-904f-75e4a9e3e724'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.070866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2906d5294c44d28b6635bc42a43d4b', 'request_id': 'req-88f8432a-617f-4079-904f-75e4a9e3e724'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.078197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2906d5294c44d28b6635bc42a43d4b', 'request_id': 'req-88f8432a-617f-4079-904f-75e4a9e3e724'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.088987 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] The token cache is being invalidate because project 2c2906d5294c44d28b6635bc42a43d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:39.089273 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.089890 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.091356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1217/2436] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:38 2022] DELETE /identity/v3/projects/2c2906d5294c44d28b6635bc42a43d4b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:39.095341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:39.104826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47427a75-3682-40fb-87f0-52f289feeb67 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:39.144922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:39.162159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:39.162786 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.163067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.163341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] PATH_INFO: `/v3/users/07ff7fb4aee64f9b91e7188c6d1251c3` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.164494 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ff7fb4aee64f9b91e7188c6d1251c3)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:39.166456 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:39.211209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ff7fb4aee64f9b91e7188c6d1251c3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.211867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ff7fb4aee64f9b91e7188c6d1251c3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.212513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ff7fb4aee64f9b91e7188c6d1251c3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.219195 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ff7fb4aee64f9b91e7188c6d1251c3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.226652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-741ee9ce-86c3-4e95-a6d4-8880cae7c34e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ff7fb4aee64f9b91e7188c6d1251c3'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.244125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1220/2437] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:39 2022] DELETE /identity/v3/users/07ff7fb4aee64f9b91e7188c6d1251c3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:39.509779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:39.511438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:39.524722 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8432a-617f-4079-904f-75e4a9e3e724 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:39.595834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:39.596484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.596764 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.597038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] PATH_INFO: `/v3/projects/cbd76cc801e043cdb3591b9d99dcafda` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.598167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd76cc801e043cdb3591b9d99dcafda)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:39.600705 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:39.648675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd76cc801e043cdb3591b9d99dcafda', 'request_id': 'req-4d797e16-a07d-47ea-9d26-444d52ecb1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.655043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd76cc801e043cdb3591b9d99dcafda', 'request_id': 'req-4d797e16-a07d-47ea-9d26-444d52ecb1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.663540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd76cc801e043cdb3591b9d99dcafda', 'request_id': 'req-4d797e16-a07d-47ea-9d26-444d52ecb1c7'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.674352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] The token cache is being invalidate because project cbd76cc801e043cdb3591b9d99dcafda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:39.674636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.675275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4d797e16-a07d-47ea-9d26-444d52ecb1c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:39.676887 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1218/2438] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:39 2022] DELETE /identity/v3/projects/cbd76cc801e043cdb3591b9d99dcafda => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:39.681881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402752-fe41-4ead-904e-019423cecaea None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.682369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402752-fe41-4ead-904e-019423cecaea None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.682462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402752-fe41-4ead-904e-019423cecaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.703578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-1d402752-fe41-4ead-904e-019423cecaea None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:39.704702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1d402752-fe41-4ead-904e-019423cecaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:39.765933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1221/2439] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:39.774355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.774668 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.774969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.796681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:39.797904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:39.806680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1219/2440] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:39.811484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402752-fe41-4ead-904e-019423cecaea None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:39.821765 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402752-fe41-4ead-904e-019423cecaea None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:39.827235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:39.827987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.828298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.828573 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.829254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:39.829558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:39.831285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:39.844259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1222/2441] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:39.852822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:39.862529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f62e52-f82e-4a5c-a16e-0a21d4bb2f0d None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:39.869344 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:39.870362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.870362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.870525 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.871091 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:39.873142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:39.878746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:39.916451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1220/2442] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:39 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:39.916907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:39.921184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed0d11-9ce6-48bd-8035-392577d109f6 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:39.968437 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:39.982043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:39.982647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:39.982930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:39.983202 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:39.983876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:39.985794 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.006842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1223/2443] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:39 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:40.011164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.021847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f99eb-3d56-44cd-a8c4-5d381a4f4a09 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.026506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.027102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.027378 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.027681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.028316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.029904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.039797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1221/2444] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:40.043662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.053866 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c46e71-b0ee-45ce-98ec-e2b1e41f87ac admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.058387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204022f-e26a-4bf2-9202-4c67585b945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.058973 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204022f-e26a-4bf2-9202-4c67585b945f admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.059248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204022f-e26a-4bf2-9202-4c67585b945f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.059557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204022f-e26a-4bf2-9202-4c67585b945f admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.060215 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0204022f-e26a-4bf2-9202-4c67585b945f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.061770 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0204022f-e26a-4bf2-9202-4c67585b945f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.072742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1224/2445] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:40.076604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.086345 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84d48-8ef0-4dff-915c-a4453383e5ad admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.090915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.091588 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.091895 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.092163 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] PATH_INFO: `/v3/projects/c4d322c9d7ef4e30b7a0dac9c3835541/users/4bfbad30890946c09c4804cde58d552b/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.106298 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d322c9d7ef4e30b7a0dac9c3835541, user_id=4bfbad30890946c09c4804cde58d552b, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.108970 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.122166 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1222/2446] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:40 2022] PUT /identity/v3/projects/c4d322c9d7ef4e30b7a0dac9c3835541/users/4bfbad30890946c09c4804cde58d552b/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:40.133273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.133557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.133848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.162170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] MFA Rules not processed for user `4bfbad30890946c09c4804cde58d552b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:40.163354 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:40.226987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1225/2447] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:40.231621 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.295299 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee57731-e226-4bda-9759-41d2e5b07cc3 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.299912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.300506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.300779 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.301047 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.301669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.304183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.332548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1223/2448] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:40 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:40.333667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.343940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f02bf-a33e-418b-8832-9de3df1e7afa None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.398182 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.398777 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.399049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.399314 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.399968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.401870 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.423925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1226/2449] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:40 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:40.428523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.439048 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e826da7-3fd4-4a1a-95e3-f36abb898c39 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.443680 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.444275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.444556 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.444830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.445471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.447052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.457460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1224/2450] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:40.461446 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.472052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012bbb9-47a3-4406-bfc6-3624f199da2e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.477268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.477920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.478204 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.478479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] PATH_INFO: `/v3/projects/15e11f115a58412b8ca007fb09114a3e/users/d0030a0a2f3049478d0ba6b57a4ec80c/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.494245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e11f115a58412b8ca007fb09114a3e, user_id=d0030a0a2f3049478d0ba6b57a4ec80c, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.496958 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.510059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1227/2451] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:40 2022] PUT /identity/v3/projects/15e11f115a58412b8ca007fb09114a3e/users/d0030a0a2f3049478d0ba6b57a4ec80c/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:40.513957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.523948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d026-3c2d-4718-bd7e-c8343b3a5c4f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.578459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40414b1-f811-4030-a803-fd613796d286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.578990 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40414b1-f811-4030-a803-fd613796d286 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.579266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40414b1-f811-4030-a803-fd613796d286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.579570 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40414b1-f811-4030-a803-fd613796d286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.580208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40414b1-f811-4030-a803-fd613796d286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.581797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40414b1-f811-4030-a803-fd613796d286 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.591821 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1225/2452] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:40.595807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.606308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ebd1a-0ddb-4ab1-88ef-2a9d7edfb9be admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.610892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.611574 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.611855 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.612389 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] PATH_INFO: `/v3/projects/15e11f115a58412b8ca007fb09114a3e/users/d0030a0a2f3049478d0ba6b57a4ec80c/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.614270 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e11f115a58412b8ca007fb09114a3e, user_id=d0030a0a2f3049478d0ba6b57a4ec80c, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.616950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.626771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1228/2453] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:40 2022] PUT /identity/v3/projects/15e11f115a58412b8ca007fb09114a3e/users/d0030a0a2f3049478d0ba6b57a4ec80c/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:40.638126 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819101-6b5c-4f04-b623-562241594916 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.638416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819101-6b5c-4f04-b623-562241594916 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.638699 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819101-6b5c-4f04-b623-562241594916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.667205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-9f819101-6b5c-4f04-b623-562241594916 None None] MFA Rules not processed for user `d0030a0a2f3049478d0ba6b57a4ec80c`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:40.668537 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9f819101-6b5c-4f04-b623-562241594916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:40.678458 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:40.739063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1226/2454] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:40.751948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.753751 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:40.764942 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:40.815107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4a725-cb18-4605-9098-140fcdb37809 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.818756 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.819368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.819681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.819950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:40.829920 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:40.830840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:40.841027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1229/2455] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:40.923546 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f819101-6b5c-4f04-b623-562241594916 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:40.934757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f819101-6b5c-4f04-b623-562241594916 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:40.938573 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:40.939277 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:40.939678 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:40.939867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:41.003466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:41.004912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:41.014984 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1227/2456] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:41.029147 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:41.030517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:41.041840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee660479-0c5b-4930-a7fc-68ccddc0b197 service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:41.045490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:41.046071 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:41.046343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:41.046607 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:41.048356 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:41.110611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:41.112324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce963157-dd64-4b4a-921d-8e17a29f9fad service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:41.121857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1230/2457] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:45.706079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:45.766579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a95f4-e81e-4fa4-95bf-c563078a7eae service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:45.771145 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:45.771806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:45.772083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:45.772352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] PATH_INFO: `/v3/users/4bfbad30890946c09c4804cde58d552b` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:45.773465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bfbad30890946c09c4804cde58d552b)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:45.775364 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:45.821641 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bfbad30890946c09c4804cde58d552b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:45.822198 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bfbad30890946c09c4804cde58d552b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:45.823354 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:45.831112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bfbad30890946c09c4804cde58d552b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:45.831860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bfbad30890946c09c4804cde58d552b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:45.839490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bfbad30890946c09c4804cde58d552b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:45.856160 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1228/2458] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:45 2022] DELETE /identity/v3/users/4bfbad30890946c09c4804cde58d552b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:45.860581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:45.861035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:45.861123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:45.880823 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:45.882008 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:45.939453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1231/2459] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:45.949281 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:45.950681 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.006912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c92923-420c-414e-94da-6b26a2f08ad4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.010614 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997aa191-c895-4f0b-9dda-8e9a1955e317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.011216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa191-c895-4f0b-9dda-8e9a1955e317 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.011528 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa191-c895-4f0b-9dda-8e9a1955e317 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.012018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa191-c895-4f0b-9dda-8e9a1955e317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.021648 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aa191-c895-4f0b-9dda-8e9a1955e317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.023077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997aa191-c895-4f0b-9dda-8e9a1955e317 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.032714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1229/2460] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:46.033165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.034560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.047022 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683c58c-58aa-449d-85a2-c347318675d5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.051685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.052315 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.052596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.052872 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] PATH_INFO: `/v3/users/c69d2c5326e048538853b6e2a241578f` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.054003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69d2c5326e048538853b6e2a241578f)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.055969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.100398 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:46.106212 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69d2c5326e048538853b6e2a241578f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.107100 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69d2c5326e048538853b6e2a241578f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.108165 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69d2c5326e048538853b6e2a241578f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.115408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69d2c5326e048538853b6e2a241578f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.122824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69d2c5326e048538853b6e2a241578f'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.141744 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1232/2461] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:46 2022] DELETE /identity/v3/users/c69d2c5326e048538853b6e2a241578f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:46.146845 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.147165 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.147483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.168415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:46.169642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.229627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1230/2462] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:46.239396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.240836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.303820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab848488-ab1f-434a-bd73-a95f95a896f9 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.307659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.308268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.308545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.309198 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.310265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.311929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.320221 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.322286 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.325210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bc2ae-ad4c-487b-8ef6-1b375a60e5b2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.330797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.331577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.331949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.332509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] PATH_INFO: `/v3/projects/c4d322c9d7ef4e30b7a0dac9c3835541` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.332619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1233/2463] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:46.334315 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d322c9d7ef4e30b7a0dac9c3835541)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.336376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.387016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d322c9d7ef4e30b7a0dac9c3835541', 'request_id': 'req-e1ceceb4-2963-4048-a0c3-fa335615d3ba'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.393667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d322c9d7ef4e30b7a0dac9c3835541', 'request_id': 'req-e1ceceb4-2963-4048-a0c3-fa335615d3ba'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.403542 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d322c9d7ef4e30b7a0dac9c3835541', 'request_id': 'req-e1ceceb4-2963-4048-a0c3-fa335615d3ba'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.405913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:46.417693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] The token cache is being invalidate because project c4d322c9d7ef4e30b7a0dac9c3835541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:46.418101 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.419051 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.420781 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1231/2464] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:46 2022] DELETE /identity/v3/projects/c4d322c9d7ef4e30b7a0dac9c3835541 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:46.424730 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.434838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf44b36-2d0a-4e37-b820-a00fefd62fe3 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.497959 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.498598 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.498876 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.499150 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] PATH_INFO: `/v3/users/d0030a0a2f3049478d0ba6b57a4ec80c` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.500338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0030a0a2f3049478d0ba6b57a4ec80c)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.502261 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.550734 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0030a0a2f3049478d0ba6b57a4ec80c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.551396 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0030a0a2f3049478d0ba6b57a4ec80c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.552591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0030a0a2f3049478d0ba6b57a4ec80c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.559316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0030a0a2f3049478d0ba6b57a4ec80c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.566999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0030a0a2f3049478d0ba6b57a4ec80c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.568534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:46.588869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1234/2465] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:46 2022] DELETE /identity/v3/users/d0030a0a2f3049478d0ba6b57a4ec80c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:46.615082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.616993 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.631367 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceceb4-2963-4048-a0c3-fa335615d3ba admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.701882 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.702647 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.703011 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.703358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] PATH_INFO: `/v3/projects/3caac8dc98444437864f9b0b413cdf8f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.704596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3caac8dc98444437864f9b0b413cdf8f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.706708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.759729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3caac8dc98444437864f9b0b413cdf8f', 'request_id': 'req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.766269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3caac8dc98444437864f9b0b413cdf8f', 'request_id': 'req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.775167 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3caac8dc98444437864f9b0b413cdf8f', 'request_id': 'req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.785599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] The token cache is being invalidate because project 3caac8dc98444437864f9b0b413cdf8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:46.785875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.787082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.788719 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1232/2466] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:46 2022] DELETE /identity/v3/projects/3caac8dc98444437864f9b0b413cdf8f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:46.792742 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.794138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.806410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e1033-5b7b-4990-b4f5-80721461d5ef admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.862849 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.863627 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.863901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.864167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] PATH_INFO: `/v3/users/fc3a5d55c44c4397861383f3c58ba547` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.865323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3a5d55c44c4397861383f3c58ba547)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.867523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.884531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:46.886253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:46.903139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c40e4-fa9b-440a-8348-c1aee8ac532a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:46.909061 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:46.909866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:46.910220 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:46.910562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] PATH_INFO: `/v3/projects/15e11f115a58412b8ca007fb09114a3e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:46.912294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e11f115a58412b8ca007fb09114a3e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:46.914238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:46.924066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3a5d55c44c4397861383f3c58ba547'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.924764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3a5d55c44c4397861383f3c58ba547'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.925367 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3a5d55c44c4397861383f3c58ba547'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.932394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3a5d55c44c4397861383f3c58ba547'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.940284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c938252d-bddc-41c8-bb39-f4873e7ec8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3a5d55c44c4397861383f3c58ba547'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.960344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1235/2467] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:46 2022] DELETE /identity/v3/users/fc3a5d55c44c4397861383f3c58ba547 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:46.967289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e11f115a58412b8ca007fb09114a3e', 'request_id': 'req-847828ad-ba22-4076-a84e-9683edbc8cc6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.974768 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e11f115a58412b8ca007fb09114a3e', 'request_id': 'req-847828ad-ba22-4076-a84e-9683edbc8cc6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:46.989352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e11f115a58412b8ca007fb09114a3e', 'request_id': 'req-847828ad-ba22-4076-a84e-9683edbc8cc6'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.000742 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] The token cache is being invalidate because project 15e11f115a58412b8ca007fb09114a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:47.001189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.001949 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-847828ad-ba22-4076-a84e-9683edbc8cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.003603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1233/2468] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:46 2022] DELETE /identity/v3/projects/15e11f115a58412b8ca007fb09114a3e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:47.009002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.009283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.009551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.030769 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:47.032101 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.086861 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:47.097036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1236/2469] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:47.104952 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.105238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.105515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.115907 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:47.136055 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:47.137461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.146283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1234/2470] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:47.150434 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.160501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260b780-8478-4ba5-a567-a6f98c73fad8 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.166762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.167551 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.167836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.168117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.168766 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:47.169069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.170654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.184069 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1237/2471] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:47.188401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.198225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0bf2a-454a-4ead-8686-e1d5517561a9 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.203361 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.204014 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.204296 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.204571 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.205519 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.207184 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.239461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1235/2472] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:47.241865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.252024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.253662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734547d2-c7e4-4a64-b956-53e27d7b7cbb admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.253946 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.314536 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.315234 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.315619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.315963 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.316600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.318581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.333853 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d7f24-0b19-4a5e-aa7e-e4636144f891 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.339330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.339974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.340249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.340517 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] PATH_INFO: `/v3/projects/c07dbe04087f431daef06d5f153e9355` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.340674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1238/2473] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:47.344470 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.344603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07dbe04087f431daef06d5f153e9355)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.346524 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.354868 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a79d2-d7de-4e95-9154-715fe1bfe910 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.359892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.360469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.360746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.361167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.361828 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.363683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.374250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1239/2474] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:47.378089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.387743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9f2d-85d9-4cc1-918e-d7835949a85f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.396499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07dbe04087f431daef06d5f153e9355', 'request_id': 'req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.406824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07dbe04087f431daef06d5f153e9355', 'request_id': 'req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.417670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07dbe04087f431daef06d5f153e9355', 'request_id': 'req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.429973 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] The token cache is being invalidate because project c07dbe04087f431daef06d5f153e9355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:47.430265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.431223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-7eff4f53-6fe5-431f-a2fd-0d5db15a364a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:47.432759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1236/2475] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:47 2022] DELETE /identity/v3/projects/c07dbe04087f431daef06d5f153e9355 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:47.438875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebd893-867f-49c6-bc7f-44466c5d1820 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.439232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebd893-867f-49c6-bc7f-44466c5d1820 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.439610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebd893-867f-49c6-bc7f-44466c5d1820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.445969 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.446579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.446858 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.447161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.447838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.449420 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.459237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-25ebd893-867f-49c6-bc7f-44466c5d1820 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:47.459827 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1240/2476] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:47.460477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25ebd893-867f-49c6-bc7f-44466c5d1820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.463956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.465527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.468430 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1237/2477] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:47.476617 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.476898 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.477175 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.478762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dbac1-7f17-4bdd-ab3e-ddd4d1a65c5b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.483466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773468c1-44e0-4a73-b588-d33fa8f4fe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.484102 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773468c1-44e0-4a73-b588-d33fa8f4fe43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.484384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773468c1-44e0-4a73-b588-d33fa8f4fe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.484654 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773468c1-44e0-4a73-b588-d33fa8f4fe43 admin admin] PATH_INFO: `/v3/projects/19981b6209934fd981cedd969c787220/users/18a2ab912d034a4ebb8cd5bd7dfb47a6/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.496465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:47.497824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.499629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773468c1-44e0-4a73-b588-d33fa8f4fe43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19981b6209934fd981cedd969c787220, user_id=18a2ab912d034a4ebb8cd5bd7dfb47a6, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.502399 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773468c1-44e0-4a73-b588-d33fa8f4fe43 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.506329 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1238/2478] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:47.510471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.518404 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1241/2479] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:47 2022] PUT /identity/v3/projects/19981b6209934fd981cedd969c787220/users/18a2ab912d034a4ebb8cd5bd7dfb47a6/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:47.519585 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea296206-7704-4a29-aaf8-71d666aebf63 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.529228 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.529519 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.529795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.557469 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] MFA Rules not processed for user `18a2ab912d034a4ebb8cd5bd7dfb47a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:47.558675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:47.570078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:47.580162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.581057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.581638 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.582121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.582848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:47.583741 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.585420 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.586030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:47.597965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1239/2480] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 07:19:47.602913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.613660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef6ee2-480a-4984-a2a3-2100a32384ea admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.618446 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.619098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.619383 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.619687 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.620316 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.622205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.626633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1242/2481] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:47.631079 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.642012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecfbcc-e12f-451c-a98a-e6bf2c9dce13 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.647341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.647916 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.648192 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.648464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.649087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.653366 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1240/2482] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:47.653497 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.654321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.681074 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1243/2483] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:47.682306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.721040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899592f-7521-4032-a9cb-57d0399255d9 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.749237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5b7d-bda5-40b4-ac5b-12ab0ce443e0 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.754132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.754739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.755606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.755937 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.756577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.758680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.780057 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1244/2484] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:47.783936 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.792769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.793327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46af9ed-170d-4a48-8931-797cd140701e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.793424 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.793657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.793924 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.794555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.796540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.797947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.798523 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.798799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.799067 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.799732 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.801543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.811981 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1245/2485] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:47.816196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.820789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1241/2486] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:47 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:19:47.824478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.828669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b55-4a5b-4ea7-a574-269e6b4a1a90 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.833548 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.834183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.834291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f9b2-4097-4b6c-9f56-a896f69219d7 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.834458 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.834726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] PATH_INFO: `/v3/projects/47721ace5cb84af5b3c7afa147c6798e/users/a3bc14c0ee0b441ebfd0fbca254807f0/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.839389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.840003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.840287 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.840562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.841202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.842794 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.850568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47721ace5cb84af5b3c7afa147c6798e, user_id=a3bc14c0ee0b441ebfd0fbca254807f0, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.853449 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.854085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1242/2487] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:47.857913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.867898 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1246/2488] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:47 2022] PUT /identity/v3/projects/47721ace5cb84af5b3c7afa147c6798e/users/a3bc14c0ee0b441ebfd0fbca254807f0/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:47.868858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd61f05-def0-42c2-8031-83dadfa8bb3a admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.871875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.915138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:47.926563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.927746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.928028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.928301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.928942 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.930558 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.934715 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb3604-872f-45f3-a31e-e2043131d212 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.939657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.940266 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.940543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.940815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.943483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1243/2489] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:47.943619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.943619 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.944681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:47.945216 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.955483 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13978e2b-c9ce-4864-b4b0-585f38262551 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.958235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1247/2490] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:47.959254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:47.960875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ab048-6b78-4d1e-87ae-908a94507c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.961523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ab048-6b78-4d1e-87ae-908a94507c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.961798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ab048-6b78-4d1e-87ae-908a94507c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.962120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ab048-6b78-4d1e-87ae-908a94507c7e admin admin] PATH_INFO: `/v3/projects/2ecc72091c5c477e80807fae97a4723c/users/eef06c38aa034d53b04a2d2ed414e92e/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.968912 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa214e2f-f7f6-4580-a59d-38fcdf39a5e1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:47.974299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea6b01-591e-4fa7-b7bc-0c61e91332b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:47.975327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea6b01-591e-4fa7-b7bc-0c61e91332b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:47.975327 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea6b01-591e-4fa7-b7bc-0c61e91332b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:47.975651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea6b01-591e-4fa7-b7bc-0c61e91332b0 admin admin] PATH_INFO: `/v3/projects/47721ace5cb84af5b3c7afa147c6798e/users/a3bc14c0ee0b441ebfd0fbca254807f0/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:47.977545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea6b01-591e-4fa7-b7bc-0c61e91332b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47721ace5cb84af5b3c7afa147c6798e, user_id=a3bc14c0ee0b441ebfd0fbca254807f0, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.977670 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ab048-6b78-4d1e-87ae-908a94507c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecc72091c5c477e80807fae97a4723c, user_id=eef06c38aa034d53b04a2d2ed414e92e, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:47.980444 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea6b01-591e-4fa7-b7bc-0c61e91332b0 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.980893 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ab048-6b78-4d1e-87ae-908a94507c7e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:47.993483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1248/2491] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:47 2022] PUT /identity/v3/projects/47721ace5cb84af5b3c7afa147c6798e/users/a3bc14c0ee0b441ebfd0fbca254807f0/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:47.995849 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1244/2492] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:47 2022] PUT /identity/v3/projects/2ecc72091c5c477e80807fae97a4723c/users/eef06c38aa034d53b04a2d2ed414e92e/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:48.006919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.007203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.007527 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.007721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.008030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.008308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.035677 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] MFA Rules not processed for user `a3bc14c0ee0b441ebfd0fbca254807f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:48.035880 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] MFA Rules not processed for user `eef06c38aa034d53b04a2d2ed414e92e`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:48.037451 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.037566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.102225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1245/2493] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:48.106579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1249/2494] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:48.114066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.115510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.117324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.118651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.172285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6502027-eee7-4d84-87be-f02a2e1aa877 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.174453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddb3bd-5027-4013-a2b2-5da1b61b5f11 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.175838 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.176442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.177223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.177499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.178633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee11bc-fc46-4134-a94f-94c1aa619a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.179267 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee11bc-fc46-4134-a94f-94c1aa619a56 service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.179572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee11bc-fc46-4134-a94f-94c1aa619a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.179843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee11bc-fc46-4134-a94f-94c1aa619a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.185989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.187437 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.188844 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee11bc-fc46-4134-a94f-94c1aa619a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.190251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee11bc-fc46-4134-a94f-94c1aa619a56 service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.196294 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1246/2495] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:48.199864 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1250/2496] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:48.219659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.283007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:48.289807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f68aa-3756-46b6-b8be-1dba711c9d0c service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.294859 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.295513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.295793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.296063 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] PATH_INFO: `/v3/users/eef06c38aa034d53b04a2d2ed414e92e` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.297191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef06c38aa034d53b04a2d2ed414e92e)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.301438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.349941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef06c38aa034d53b04a2d2ed414e92e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.350604 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef06c38aa034d53b04a2d2ed414e92e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.357693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef06c38aa034d53b04a2d2ed414e92e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.358338 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef06c38aa034d53b04a2d2ed414e92e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.370066 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef06c38aa034d53b04a2d2ed414e92e'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.394135 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1247/2497] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:48 2022] DELETE /identity/v3/users/eef06c38aa034d53b04a2d2ed414e92e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:48.399154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.399704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.399800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.425464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:48.426642 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.491956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1251/2498] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:48.502426 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.503876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.569899 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5a185-94fa-47f4-8961-cc7e8c53e96e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.573657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6158c16-5437-4f2f-a431-8b2c02f144d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.574266 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6158c16-5437-4f2f-a431-8b2c02f144d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.574545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6158c16-5437-4f2f-a431-8b2c02f144d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.574815 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6158c16-5437-4f2f-a431-8b2c02f144d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.589603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6158c16-5437-4f2f-a431-8b2c02f144d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.591057 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6158c16-5437-4f2f-a431-8b2c02f144d9 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.594599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.596140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.601026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1248/2499] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:48.610048 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32f95-0017-45e4-96f1-fb9c1176d71d None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.614940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.615586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.615867 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.616132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] PATH_INFO: `/v3/users/18a2ab912d034a4ebb8cd5bd7dfb47a6` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.617277 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a2ab912d034a4ebb8cd5bd7dfb47a6)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.619176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.623840 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:48.670391 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a2ab912d034a4ebb8cd5bd7dfb47a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.671091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a2ab912d034a4ebb8cd5bd7dfb47a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.671738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a2ab912d034a4ebb8cd5bd7dfb47a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.678933 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a2ab912d034a4ebb8cd5bd7dfb47a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.686357 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a2ab912d034a4ebb8cd5bd7dfb47a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.705283 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1252/2500] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:48 2022] DELETE /identity/v3/users/18a2ab912d034a4ebb8cd5bd7dfb47a6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:48.710411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.710411 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.710693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.730767 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:48.732002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.799179 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1249/2501] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:48.809279 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.810850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.870358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:48.872387 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:48.873445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53796d0-4202-45f1-a78e-ff46d8d5dce1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.877163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91c7a7-fa72-44a4-a12d-979371e8c564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.878422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91c7a7-fa72-44a4-a12d-979371e8c564 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.878704 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91c7a7-fa72-44a4-a12d-979371e8c564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.878976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91c7a7-fa72-44a4-a12d-979371e8c564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.885675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917ba5d-1a77-4ff5-bda8-dffe1bbfb731 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:48.890415 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:48.891041 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.891323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.891671 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] PATH_INFO: `/v3/projects/2ecc72091c5c477e80807fae97a4723c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:48.892860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ecc72091c5c477e80807fae97a4723c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.894710 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.901776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91c7a7-fa72-44a4-a12d-979371e8c564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:48.903343 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91c7a7-fa72-44a4-a12d-979371e8c564 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:48.907822 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:48.913927 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1253/2502] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:48.946009 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ecc72091c5c477e80807fae97a4723c', 'request_id': 'req-aefc7037-8739-415a-9dea-82e3d78921df'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.952254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ecc72091c5c477e80807fae97a4723c', 'request_id': 'req-aefc7037-8739-415a-9dea-82e3d78921df'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.960320 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ecc72091c5c477e80807fae97a4723c', 'request_id': 'req-aefc7037-8739-415a-9dea-82e3d78921df'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.972032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] The token cache is being invalidate because project 2ecc72091c5c477e80807fae97a4723c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:48.972324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.972974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-aefc7037-8739-415a-9dea-82e3d78921df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:48.974696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1250/2503] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:48 2022] DELETE /identity/v3/projects/2ecc72091c5c477e80807fae97a4723c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:48.979628 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:48.979918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:48.980193 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.001126 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:49.002339 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.063135 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1254/2504] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:49.073015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.073307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.073624 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.093688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:49.094910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.103033 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1251/2505] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:49.107359 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.118252 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695ee64-2132-4316-9512-f52eaaaa783f None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.122901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.123531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.123813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.124087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.124735 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:49.125041 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.126625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.140509 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1255/2506] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:49.143122 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.152231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d3638-cec8-4f21-bfd6-e57ae26f7862 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.156913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.157526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.157802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.158067 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.158688 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.160605 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.189820 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1252/2507] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:49 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:49.194208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.196805 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.198445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.204992 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1c9a-6ef8-4206-af43-887da84398e2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.215216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:49.269569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf17545-0fc8-4277-8486-2017d6f7c46d admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.269938 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.270675 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.270983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.271254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.271917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.273814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.274333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.274965 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.275248 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.275557 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] PATH_INFO: `/v3/projects/19981b6209934fd981cedd969c787220` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.276693 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19981b6209934fd981cedd969c787220)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.278591 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.295878 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1256/2508] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:49 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:49.299764 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.309223 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6dbd5-1255-4a8c-b287-fc9f6316a1b6 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.313939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.314538 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.314820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.315094 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.315919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.318036 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.328139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19981b6209934fd981cedd969c787220', 'request_id': 'req-c1212731-10f3-4d70-9856-ed509cc2110b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.328306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1257/2509] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:49.331901 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.334289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19981b6209934fd981cedd969c787220', 'request_id': 'req-c1212731-10f3-4d70-9856-ed509cc2110b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.341667 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19981b6209934fd981cedd969c787220', 'request_id': 'req-c1212731-10f3-4d70-9856-ed509cc2110b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.342290 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fbb57-6b1d-489a-893b-03d3662d024e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.353323 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] The token cache is being invalidate because project 19981b6209934fd981cedd969c787220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:49.353616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.354330 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.356136 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1253/2510] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:49 2022] DELETE /identity/v3/projects/19981b6209934fd981cedd969c787220 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:49.359642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.360960 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.366240 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:49.405956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.405956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.405956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.405956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.406448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.408366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.418288 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1258/2511] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:49.422415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.424085 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.432241 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1212731-10f3-4d70-9856-ed509cc2110b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.436021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20300806-f1a1-4cb6-9e04-ee4774bd7881 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.437460 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.438093 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.438376 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.438651 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] PATH_INFO: `/v3/users/a3bc14c0ee0b441ebfd0fbca254807f0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.439808 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3bc14c0ee0b441ebfd0fbca254807f0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.440546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5550dc1-3764-4db6-a5d4-60d0b445d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.441173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5550dc1-3764-4db6-a5d4-60d0b445d02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.441448 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5550dc1-3764-4db6-a5d4-60d0b445d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.441718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5550dc1-3764-4db6-a5d4-60d0b445d02d admin admin] PATH_INFO: `/v3/projects/92c63a2b8e024416923cc53147c23a4d/users/a791f7a388f444ae837ff23e7b1cd212/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.441929 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.458589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5550dc1-3764-4db6-a5d4-60d0b445d02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c63a2b8e024416923cc53147c23a4d, user_id=a791f7a388f444ae837ff23e7b1cd212, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.462545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5550dc1-3764-4db6-a5d4-60d0b445d02d admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.476633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1259/2512] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:49 2022] PUT /identity/v3/projects/92c63a2b8e024416923cc53147c23a4d/users/a791f7a388f444ae837ff23e7b1cd212/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:49.487138 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.487467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.487881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.491256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3bc14c0ee0b441ebfd0fbca254807f0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.491954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3bc14c0ee0b441ebfd0fbca254807f0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.499301 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3bc14c0ee0b441ebfd0fbca254807f0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.500119 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3bc14c0ee0b441ebfd0fbca254807f0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.508748 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3bc14c0ee0b441ebfd0fbca254807f0'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.516244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] MFA Rules not processed for user `a791f7a388f444ae837ff23e7b1cd212`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:49.517489 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.526480 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1254/2513] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:49 2022] DELETE /identity/v3/users/a3bc14c0ee0b441ebfd0fbca254807f0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:49.578781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:49.592081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1260/2514] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:49.604339 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.606254 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.666630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f65e10-b936-49c0-8e30-9527d0310c3b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.670860 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c1421-66de-4274-a896-385e2705cc6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.671291 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1421-66de-4274-a896-385e2705cc6d service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.671626 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1421-66de-4274-a896-385e2705cc6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.671871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1421-66de-4274-a896-385e2705cc6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.709015 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:49.734082 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c1421-66de-4274-a896-385e2705cc6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.735579 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c1421-66de-4274-a896-385e2705cc6d service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.745499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1255/2515] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:49.803613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:49.805213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:49.878247 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959cc8-1dea-4f98-8d11-631974064a85 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:49.937731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:49.938520 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:49.938843 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:49.939154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] PATH_INFO: `/v3/projects/47721ace5cb84af5b3c7afa147c6798e` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:49.940334 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47721ace5cb84af5b3c7afa147c6798e)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:49.942753 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:49.989966 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47721ace5cb84af5b3c7afa147c6798e', 'request_id': 'req-e3063b22-1a94-4d69-82d0-d1e2848592f2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:49.999301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47721ace5cb84af5b3c7afa147c6798e', 'request_id': 'req-e3063b22-1a94-4d69-82d0-d1e2848592f2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:50.008300 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47721ace5cb84af5b3c7afa147c6798e', 'request_id': 'req-e3063b22-1a94-4d69-82d0-d1e2848592f2'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:50.019271 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] The token cache is being invalidate because project 47721ace5cb84af5b3c7afa147c6798e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:50.019462 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:50.020173 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-e3063b22-1a94-4d69-82d0-d1e2848592f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:50.022213 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1261/2516] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:49 2022] DELETE /identity/v3/projects/47721ace5cb84af5b3c7afa147c6798e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:50.027153 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.027468 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.027738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.048761 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:50.049997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:50.125743 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1256/2517] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:50.133244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.133531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.133810 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.153658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:50.154846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:50.163163 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1262/2518] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:50.167265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.177575 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7a65-5a6b-48d2-84e1-5a62ea0cbf29 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.182419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.183677 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.184007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.184349 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] PATH_INFO: `/v3/domains` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.185078 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81485) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:50.185379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.186980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.198875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1257/2519] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:19:50.203087 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.212209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b0a3e-ba56-4d77-82b0-c899c3c3650c None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.216854 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.217450 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.217729 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.218002 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.218641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.220631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.252408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1263/2520] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:50 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:50.253535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.263989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e7a8-80ec-4068-89c4-2fb718203011 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.320130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.320733 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.321012 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.321286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.321914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.323874 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.344365 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1258/2521] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:50 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:50.348374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.353746 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:50.359839 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72094ec-02cc-41d5-ab79-abe493615d82 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.366089 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.366680 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.366970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.367241 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.367906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.369479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.382791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1264/2522] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:50.385379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.395633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f310a-19e9-44a5-acce-6cce486f1310 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.400303 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3adcb-b99a-4c3e-9db8-eb534a56215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.400903 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3adcb-b99a-4c3e-9db8-eb534a56215b admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.401183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3adcb-b99a-4c3e-9db8-eb534a56215b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.401463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3adcb-b99a-4c3e-9db8-eb534a56215b admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.402106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3adcb-b99a-4c3e-9db8-eb534a56215b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.403744 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3adcb-b99a-4c3e-9db8-eb534a56215b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.414858 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1259/2523] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:50.419018 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.429710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add67be-2881-412b-8420-115e1f03506a admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.434834 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.435512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.435790 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.436059 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] PATH_INFO: `/v3/projects/7ff6cc68ca35415692c19f27348ad6d0/users/19004f15d0b948cd90b254b7d5ff11a6/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.452055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff6cc68ca35415692c19f27348ad6d0, user_id=19004f15d0b948cd90b254b7d5ff11a6, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.454738 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.466799 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1265/2524] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:50 2022] PUT /identity/v3/projects/7ff6cc68ca35415692c19f27348ad6d0/users/19004f15d0b948cd90b254b7d5ff11a6/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:50.471796 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25582c56-a180-4e9d-9911-0dd8d28dccfd None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.472025 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25582c56-a180-4e9d-9911-0dd8d28dccfd None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.472300 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25582c56-a180-4e9d-9911-0dd8d28dccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.491506 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-25582c56-a180-4e9d-9911-0dd8d28dccfd None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:50.492698 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-25582c56-a180-4e9d-9911-0dd8d28dccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:50.529657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:50.550834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1260/2525] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:50.561255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:50.563014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:50.618757 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1574af-b57b-4c9d-a593-c37191839e3e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:50.622378 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:50.623029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:50.623320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:50.623623 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:50.633083 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:50.634503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:50.643795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1266/2526] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:51.273620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.274332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.274332 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.304867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] MFA Rules not processed for user `19004f15d0b948cd90b254b7d5ff11a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:51.306227 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:51.383114 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1261/2527] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:51.387631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:51.398530 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11353e7-2f38-4527-8d81-edd2584b4ea6 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:51.403181 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:51.403803 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.404080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.404347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] PATH_INFO: `/v3/projects` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.405016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:51.407998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:51.443737 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1267/2528] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:51 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:51.445068 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:51.456658 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebd3c9-86fa-41e7-9c0d-6674dc145c34 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:51.518449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:51.519074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.519357 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.519669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.520917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:51.522954 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:51.544084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1262/2529] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:51 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:51.548374 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:51.559384 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4572ff-cd8a-462f-9b10-8f29e6af635e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:51.564754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:51.565394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.565674 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.570926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.570926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:51.570926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:51.581145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1268/2530] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:19:51.585969 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:51.596077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821130c2-4e9c-4fa9-b46c-750c47854d9b admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:51.601148 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:51.601798 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.602104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.602379 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] PATH_INFO: `/v3/projects/654556b3a0804225a54d7905ed13db1f/users/1c40683c1dc740b08e37eb8ba3ac33ca/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.617846 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=654556b3a0804225a54d7905ed13db1f, user_id=1c40683c1dc740b08e37eb8ba3ac33ca, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:51.620833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:51.634501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1263/2531] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:51 2022] PUT /identity/v3/projects/654556b3a0804225a54d7905ed13db1f/users/1c40683c1dc740b08e37eb8ba3ac33ca/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:51.638471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:51.649622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9756d6-f47e-4ec7-bbd8-f1a8180952d7 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:51.710209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff5ff-9457-49eb-a0fa-e147917f9399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:51.710820 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff5ff-9457-49eb-a0fa-e147917f9399 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.711095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff5ff-9457-49eb-a0fa-e147917f9399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.711363 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff5ff-9457-49eb-a0fa-e147917f9399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.712032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bff5ff-9457-49eb-a0fa-e147917f9399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:51.713589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bff5ff-9457-49eb-a0fa-e147917f9399 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:51.723647 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1269/2532] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:51.727804 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:51.739210 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf456c2b-6426-45c4-95ba-cd1c13592ccf admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:51.744401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:51.745195 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:51.745548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:51.745955 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] PATH_INFO: `/v3/projects/654556b3a0804225a54d7905ed13db1f/users/1c40683c1dc740b08e37eb8ba3ac33ca/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:51.750883 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=654556b3a0804225a54d7905ed13db1f, user_id=1c40683c1dc740b08e37eb8ba3ac33ca, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:51.753766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:51.765599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1264/2533] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:51 2022] PUT /identity/v3/projects/654556b3a0804225a54d7905ed13db1f/users/1c40683c1dc740b08e37eb8ba3ac33ca/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:52.383785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:52.384336 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:52.384621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:52.416320 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] MFA Rules not processed for user `1c40683c1dc740b08e37eb8ba3ac33ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:52.417699 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:52.439250 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:52.492140 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1270/2534] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:52.503607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:52.505477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:52.534120 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:52.567040 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0eaf4-c6c5-4de5-87ac-f1248569c7cd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:52.570576 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f807-4507-4359-8ad6-208da33b578f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:52.571183 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f807-4507-4359-8ad6-208da33b578f service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:52.571486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f807-4507-4359-8ad6-208da33b578f service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:52.571757 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f807-4507-4359-8ad6-208da33b578f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:52.629628 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6f807-4507-4359-8ad6-208da33b578f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:52.631084 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6f807-4507-4359-8ad6-208da33b578f service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:52.640422 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1265/2535] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:52.831525 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:52.833743 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:52.911611 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4750a3-b348-42c1-9b25-cb7867ecc6e3 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:52.917397 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:52.918030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:52.918308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:52.918609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] PATH_INFO: `/v3/users/a791f7a388f444ae837ff23e7b1cd212` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:52.919752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a791f7a388f444ae837ff23e7b1cd212)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:52.921688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:52.973344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a791f7a388f444ae837ff23e7b1cd212'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:52.974032 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a791f7a388f444ae837ff23e7b1cd212'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:52.974651 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a791f7a388f444ae837ff23e7b1cd212'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:52.981986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a791f7a388f444ae837ff23e7b1cd212'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:52.989060 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a791f7a388f444ae837ff23e7b1cd212'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:53.006013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1271/2536] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:19:52 2022] DELETE /identity/v3/users/a791f7a388f444ae837ff23e7b1cd212 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:53.010619 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.010904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.011177 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.031180 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:53.032423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:53.092003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1266/2537] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:53.101564 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.102948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:53.143657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:53.159815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be8939-5a00-426e-a779-97b548058b7e admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.163324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa13a0-0428-4510-bf0d-9a8747187fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.163941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa13a0-0428-4510-bf0d-9a8747187fff service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.164216 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa13a0-0428-4510-bf0d-9a8747187fff service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.164483 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa13a0-0428-4510-bf0d-9a8747187fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.173078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa13a0-0428-4510-bf0d-9a8747187fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.174517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa13a0-0428-4510-bf0d-9a8747187fff service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.183688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1272/2538] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:19:53.450695 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.452225 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:53.465540 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66779-09fa-445e-8799-355b1d9425d7 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.470866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.471696 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.472046 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.472393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] PATH_INFO: `/v3/projects/92c63a2b8e024416923cc53147c23a4d` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.473654 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c63a2b8e024416923cc53147c23a4d)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.475707 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.523285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c63a2b8e024416923cc53147c23a4d', 'request_id': 'req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:53.529461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c63a2b8e024416923cc53147c23a4d', 'request_id': 'req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:53.537089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c63a2b8e024416923cc53147c23a4d', 'request_id': 'req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:53.547660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] The token cache is being invalidate because project 92c63a2b8e024416923cc53147c23a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:19:53.547943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:53.548564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-1a5812c0-b3b7-444e-ae96-353a5bfee2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:19:53.550031 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1267/2539] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:19:53 2022] DELETE /identity/v3/projects/92c63a2b8e024416923cc53147c23a4d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:19:53.555447 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.555911 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.556043 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.575772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:53.576978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:53.634425 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1273/2540] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:53.642002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.642293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.642545 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.650077 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:53.664229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:53.665501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:53.673295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1268/2541] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:19:53.677997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.687612 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f465e-1ccd-47c9-8adf-375830ea6e95 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.692246 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.692838 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.693117 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.693388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.694029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:19:53.694330 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.696012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.707301 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1274/2542] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 07:19:53.711325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.720358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c6869-eb53-47c3-b7eb-1d6a61770236 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.725478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.726065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.726340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.726612 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.727238 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.729154 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.758304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1269/2543] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:19:53 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:53.762217 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.772863 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f77e31-0a81-4c99-9893-41eef73ddfb9 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.826337 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.826929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.827206 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.827633 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.828260 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.830139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.851210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1275/2544] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:19:53 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:19:53.855265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.865814 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83743ee0-4441-4f9b-b8b6-98009ffda7fc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.870352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.870941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.871213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.871515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.872146 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.873724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.883889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1270/2545] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:53.887608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.897755 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e60c-a706-4d44-8ee7-1f916a8f5cef admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.902229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042678ba-11e0-49df-8763-eb056d6d2b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.902806 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042678ba-11e0-49df-8763-eb056d6d2b66 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.903081 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042678ba-11e0-49df-8763-eb056d6d2b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.903349 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042678ba-11e0-49df-8763-eb056d6d2b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.903996 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042678ba-11e0-49df-8763-eb056d6d2b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.905550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042678ba-11e0-49df-8763-eb056d6d2b66 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.915773 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1276/2546] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:19:53.919773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.929384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d4c1b-05e2-417e-b16c-e3ac0d28535e admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:53.933867 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:53.934497 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.934912 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.935190 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] PATH_INFO: `/v3/projects/1d786e2dcdef4a5bb62aef943c32c18a/users/50b04df78eff4b5cbc64b76e01ff8bdd/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.951729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d786e2dcdef4a5bb62aef943c32c18a, user_id=50b04df78eff4b5cbc64b76e01ff8bdd, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:53.954372 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:53.968285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1271/2547] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:19:53 2022] PUT /identity/v3/projects/1d786e2dcdef4a5bb62aef943c32c18a/users/50b04df78eff4b5cbc64b76e01ff8bdd/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:19:53.979511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:53.979800 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:53.980076 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:53.996675 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:53.998401 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:54.009534 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] MFA Rules not processed for user `50b04df78eff4b5cbc64b76e01ff8bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:19:54.010710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:54.045191 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:19:54.058247 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39760b-1b10-4c59-83fe-cb8117344376 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:54.062158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:54.063083 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:54.063362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:54.063663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:54.065470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:54.078648 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1277/2548] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:19:54.089752 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:19:54.091110 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:19:54.102517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130e2d1-d631-4116-98d4-3165bd684851 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:19:54.106075 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:19:54.106649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:19:54.106923 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:19:54.107189 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:19:54.116383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:54.117791 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:54.126569 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1278/2549] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:19:54.135395 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:19:54.136871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:19:54.146508 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1272/2550] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:19:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:20:04.122546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=17 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:04.142087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=76,keystone:INSERT=2 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:09.256177 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:09.257826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:09.329432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46295385-cc8b-4db1-a52c-73cacf2ae2cd service cinder] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:09.335684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:09.336318 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:09.336591 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:09.336857 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] PATH_INFO: `/v3/users/19004f15d0b948cd90b254b7d5ff11a6` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:09.338424 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19004f15d0b948cd90b254b7d5ff11a6)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:09.340386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:09.389824 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19004f15d0b948cd90b254b7d5ff11a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.390476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19004f15d0b948cd90b254b7d5ff11a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.391130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19004f15d0b948cd90b254b7d5ff11a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.399476 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19004f15d0b948cd90b254b7d5ff11a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.412341 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19004f15d0b948cd90b254b7d5ff11a6'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.430592 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1279/2551] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:20:09 2022] DELETE /identity/v3/users/19004f15d0b948cd90b254b7d5ff11a6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:09.626593 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:09.644633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5cf25-7a70-4e27-931c-eb4bf5d1417e service cinder] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:09.701724 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:09.702470 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:09.702752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:09.703024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] PATH_INFO: `/v3/projects/7ff6cc68ca35415692c19f27348ad6d0` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:09.704392 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff6cc68ca35415692c19f27348ad6d0)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:09.706623 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:09.753563 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff6cc68ca35415692c19f27348ad6d0', 'request_id': 'req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.760356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff6cc68ca35415692c19f27348ad6d0', 'request_id': 'req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.768065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff6cc68ca35415692c19f27348ad6d0', 'request_id': 'req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.779312 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] The token cache is being invalidate because project 7ff6cc68ca35415692c19f27348ad6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:20:09.779657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.780292 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.781780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1273/2552] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:20:09 2022] DELETE /identity/v3/projects/7ff6cc68ca35415692c19f27348ad6d0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:09.785939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:09.787390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:09.800577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7372d-6123-49e8-aac3-84226d4f38b2 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:09.855985 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:09.856609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:09.856884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:09.857154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] PATH_INFO: `/v3/users/1c40683c1dc740b08e37eb8ba3ac33ca` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:09.858278 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c40683c1dc740b08e37eb8ba3ac33ca)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:09.860209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:09.882570 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:09.910183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c40683c1dc740b08e37eb8ba3ac33ca'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.910848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c40683c1dc740b08e37eb8ba3ac33ca'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.911475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c40683c1dc740b08e37eb8ba3ac33ca'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.918965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c40683c1dc740b08e37eb8ba3ac33ca'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.927098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-9b8be9e2-f82c-4115-9297-41bf95fcc599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c40683c1dc740b08e37eb8ba3ac33ca'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:09.946822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1280/2553] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:20:09 2022] DELETE /identity/v3/users/1c40683c1dc740b08e37eb8ba3ac33ca => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:10.122286 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.123998 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:10.137191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c56c2-7afe-4e61-93b5-c3ff4073567f admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.196873 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.197523 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.197807 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.198085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] PATH_INFO: `/v3/projects/654556b3a0804225a54d7905ed13db1f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.199231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654556b3a0804225a54d7905ed13db1f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.201239 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.235289 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:10.249026 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654556b3a0804225a54d7905ed13db1f', 'request_id': 'req-39a26d6b-b79b-43dc-bb49-189abf187d50'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:10.255336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654556b3a0804225a54d7905ed13db1f', 'request_id': 'req-39a26d6b-b79b-43dc-bb49-189abf187d50'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:10.263322 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654556b3a0804225a54d7905ed13db1f', 'request_id': 'req-39a26d6b-b79b-43dc-bb49-189abf187d50'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:10.275003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] The token cache is being invalidate because project 654556b3a0804225a54d7905ed13db1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:20:10.275293 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:10.276008 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-39a26d6b-b79b-43dc-bb49-189abf187d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:10.277493 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1274/2554] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:20:10 2022] DELETE /identity/v3/projects/654556b3a0804225a54d7905ed13db1f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:10.283120 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.283649 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.283739 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.303934 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:10.305132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:10.367972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1281/2555] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:10.376038 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.376371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.376729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.398985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:10.400156 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:10.408438 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1275/2556] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:10.413209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.423471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25255813-86d5-4f3f-8902-08a0203ba1e6 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.428829 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.429453 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.429728 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.429999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.430659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:20:10.430956 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.432585 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.444688 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1282/2557] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:20:10.448847 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.457904 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff7d6b-1a76-4157-a6d1-4a48090d07d1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.462412 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.462997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.463268 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.463578 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.464205 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.466104 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.496116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1276/2558] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:20:10 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:20:10.500310 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.510342 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c1225-47f7-4c02-88ef-a59d6bf03a5f admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.567731 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.568345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.568626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.569003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.569568 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.571505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.601109 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1283/2559] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:20:10 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:20:10.605940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.616801 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a98311-f7db-4e94-9e9e-fe16f75a6904 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.621455 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.622056 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.622359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.622634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.623275 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.624951 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.635252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1277/2560] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:10.639322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.649833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810a638-0d27-47ab-9bf8-73a28cec2522 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.654285 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1709834-e84a-4a38-96d4-7cbc39d5660f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.654879 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709834-e84a-4a38-96d4-7cbc39d5660f admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.655159 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709834-e84a-4a38-96d4-7cbc39d5660f admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.655445 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709834-e84a-4a38-96d4-7cbc39d5660f admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.656172 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1709834-e84a-4a38-96d4-7cbc39d5660f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.657741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1709834-e84a-4a38-96d4-7cbc39d5660f admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.673347 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1284/2561] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:10.674729 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.685003 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce6795-f914-4e6e-bbf1-c187aea80f10 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.689608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.690257 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.690536 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.690806 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] PATH_INFO: `/v3/projects/e020ee8a3213450bb18defd9c6e74742/users/acb5de37dbff46fdabe9111cc13ec0d5/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.705213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e020ee8a3213450bb18defd9c6e74742, user_id=acb5de37dbff46fdabe9111cc13ec0d5, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.708421 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.721260 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1278/2562] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:20:10 2022] PUT /identity/v3/projects/e020ee8a3213450bb18defd9c6e74742/users/acb5de37dbff46fdabe9111cc13ec0d5/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:10.733145 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.733556 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.733707 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.742479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:10.764055 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] MFA Rules not processed for user `acb5de37dbff46fdabe9111cc13ec0d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:10.765276 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:10.829144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1285/2563] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:10.841864 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:10.843834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:10.903229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3f0d9-d0c3-4f51-8af9-d858dda61d94 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:10.906797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d752ac7-50a9-455b-9ea5-5ea749128979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:10.907396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752ac7-50a9-455b-9ea5-5ea749128979 service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:10.907701 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752ac7-50a9-455b-9ea5-5ea749128979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:10.907974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752ac7-50a9-455b-9ea5-5ea749128979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:10.916230 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d752ac7-50a9-455b-9ea5-5ea749128979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:10.917653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d752ac7-50a9-455b-9ea5-5ea749128979 service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:10.927029 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1279/2564] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:20:10.946988 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.010667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdeb38-062b-4bb9-a0ea-22c161ae1671 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:11.015280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:11.015944 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.016220 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.016493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] PATH_INFO: `/v3/users/acb5de37dbff46fdabe9111cc13ec0d5` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.017624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb5de37dbff46fdabe9111cc13ec0d5)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:11.019579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:11.065645 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb5de37dbff46fdabe9111cc13ec0d5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.066338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb5de37dbff46fdabe9111cc13ec0d5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.066993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb5de37dbff46fdabe9111cc13ec0d5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.080016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb5de37dbff46fdabe9111cc13ec0d5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.087581 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb5de37dbff46fdabe9111cc13ec0d5'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.104658 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1286/2565] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:20:10 2022] DELETE /identity/v3/users/acb5de37dbff46fdabe9111cc13ec0d5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:11.109208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.109495 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.109769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.129325 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:11.131037 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:11.183121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:11.193780 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1280/2566] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:11.203466 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.204856 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:11.220119 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:11.264726 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947dc27-c257-4c16-949e-3dec5f507b79 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:11.268352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c404cc-b87a-4235-89eb-45fd9b047a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:11.268978 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c404cc-b87a-4235-89eb-45fd9b047a52 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.269259 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c404cc-b87a-4235-89eb-45fd9b047a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.269517 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c404cc-b87a-4235-89eb-45fd9b047a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.278822 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c404cc-b87a-4235-89eb-45fd9b047a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:11.280258 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c404cc-b87a-4235-89eb-45fd9b047a52 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:11.289557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1287/2567] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:11.559568 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.560977 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:11.585042 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb40970-8da2-4ef1-8e26-206c8da3524f None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:11.589642 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:11.590270 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.590544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.590813 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] PATH_INFO: `/v3/projects/e020ee8a3213450bb18defd9c6e74742` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.591944 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e020ee8a3213450bb18defd9c6e74742)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:11.593900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:11.642531 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e020ee8a3213450bb18defd9c6e74742', 'request_id': 'req-bff24eec-26eb-4987-8559-5db96bf31d09'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.649043 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e020ee8a3213450bb18defd9c6e74742', 'request_id': 'req-bff24eec-26eb-4987-8559-5db96bf31d09'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.657045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e020ee8a3213450bb18defd9c6e74742', 'request_id': 'req-bff24eec-26eb-4987-8559-5db96bf31d09'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.673471 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] The token cache is being invalidate because project e020ee8a3213450bb18defd9c6e74742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:20:11.673759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.674405 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-bff24eec-26eb-4987-8559-5db96bf31d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:11.675920 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1281/2568] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:20:11 2022] DELETE /identity/v3/projects/e020ee8a3213450bb18defd9c6e74742 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:11.681265 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.681546 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.681814 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.701091 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:11.702208 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:11.762245 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1288/2569] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:11.769900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.770188 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.770463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.790587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:11.791829 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:11.799663 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1282/2570] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:11.804485 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.813935 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306db8e-6d9c-43b4-98d8-a15ea557a0fb None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:11.818507 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:11.819098 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.819371 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.819667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.820298 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:20:11.820593 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:11.822190 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:11.834401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1289/2571] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 07:20:11.838787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.848094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5a3ba-9920-42f5-a37d-949d218016e1 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:11.853363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:11.853959 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.854237 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.854509 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.855141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:11.857080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:11.884168 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1283/2572] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:20:11 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:20:11.888302 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.898203 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697af498-1078-4155-ab9b-e389ee0d2ed5 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:11.955657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:11.956196 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:11.956475 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:11.956749 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:11.957401 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:11.959304 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:11.979779 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1290/2573] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:20:11 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:20:11.983888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:11.995278 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1977679-92d7-4e41-b5fc-a93b86ed9bbc admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:12.000562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:12.001196 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:12.001477 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:12.001905 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:12.002564 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:12.004213 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:12.015396 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1284/2574] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:12.019436 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:12.029881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784af5a-1e86-4f79-bbbd-1f7dc8979593 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:12.034579 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be3759-9111-4735-a7fe-5699e8256a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:12.035176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be3759-9111-4735-a7fe-5699e8256a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:12.035501 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be3759-9111-4735-a7fe-5699e8256a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:12.035781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be3759-9111-4735-a7fe-5699e8256a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:12.036994 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be3759-9111-4735-a7fe-5699e8256a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:12.038576 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be3759-9111-4735-a7fe-5699e8256a95 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:12.048913 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1291/2575] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:12.052917 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:12.062627 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc92c03-97c1-4c5c-bd29-6ae18be5ecf4 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:12.067164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:12.067830 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:12.068111 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:12.068377 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] PATH_INFO: `/v3/projects/94ce1d1c6a1a49e494ff30ad588f661c/users/06abb53024d04abbad308a8d91d09ff1/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:12.083244 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ce1d1c6a1a49e494ff30ad588f661c, user_id=06abb53024d04abbad308a8d91d09ff1, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:12.085943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:12.101019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1285/2576] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:20:12 2022] PUT /identity/v3/projects/94ce1d1c6a1a49e494ff30ad588f661c/users/06abb53024d04abbad308a8d91d09ff1/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:12.112435 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:12.112720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:12.112993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:12.139438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] MFA Rules not processed for user `06abb53024d04abbad308a8d91d09ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:12.140625 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:12.150209 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:12.209186 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1292/2577] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:12.220583 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:12.222385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:12.280121 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478578e0-5de4-4870-8369-2e194e852aec admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:12.283766 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303eb81-41ce-49ee-b81e-4b31af162322 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:12.284371 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303eb81-41ce-49ee-b81e-4b31af162322 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:12.284650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303eb81-41ce-49ee-b81e-4b31af162322 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:12.284919 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303eb81-41ce-49ee-b81e-4b31af162322 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:12.293127 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303eb81-41ce-49ee-b81e-4b31af162322 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:12.294550 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303eb81-41ce-49ee-b81e-4b31af162322 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:12.303769 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1286/2578] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:22.207474 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=22 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:22.299328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:34.620721 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:34.622112 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:34.687809 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb492a-8b59-4cea-b4d2-a4623491a53b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:34.692669 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:34.693299 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:34.693577 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:34.693850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] PATH_INFO: `/v3/users/50b04df78eff4b5cbc64b76e01ff8bdd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:34.695011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b04df78eff4b5cbc64b76e01ff8bdd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:34.696953 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:34.742479 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b04df78eff4b5cbc64b76e01ff8bdd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:34.743274 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b04df78eff4b5cbc64b76e01ff8bdd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:34.743925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b04df78eff4b5cbc64b76e01ff8bdd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:34.751254 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b04df78eff4b5cbc64b76e01ff8bdd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:34.758422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b04df78eff4b5cbc64b76e01ff8bdd'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:34.775294 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1293/2579] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:20:34 2022] DELETE /identity/v3/users/50b04df78eff4b5cbc64b76e01ff8bdd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:34.779793 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:34.780304 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:34.780410 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:34.799467 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:34.800659 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:34.860231 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1287/2580] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:20:34.870345 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:34.871778 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:34.928807 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b36cbb-adc4-4ed4-af80-48e609805378 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:34.932415 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f11cc-de99-4ffa-b15b-1ab4724040c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:34.933013 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f11cc-de99-4ffa-b15b-1ab4724040c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:34.933289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f11cc-de99-4ffa-b15b-1ab4724040c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:34.933557 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f11cc-de99-4ffa-b15b-1ab4724040c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:34.942939 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f11cc-de99-4ffa-b15b-1ab4724040c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:34.944379 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f11cc-de99-4ffa-b15b-1ab4724040c7 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:34.953634 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1294/2581] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:35.215522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.217491 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:35.229105 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e3ea4-3446-4880-b0dc-1e306834dbb6 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.234324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.234941 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.235218 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.235773 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] PATH_INFO: `/v3/projects/1d786e2dcdef4a5bb62aef943c32c18a` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.236950 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d786e2dcdef4a5bb62aef943c32c18a)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.238876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.285795 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d786e2dcdef4a5bb62aef943c32c18a', 'request_id': 'req-c8c194c7-0c58-48db-b79c-9890094c7e99'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:35.291943 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d786e2dcdef4a5bb62aef943c32c18a', 'request_id': 'req-c8c194c7-0c58-48db-b79c-9890094c7e99'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:35.299635 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d786e2dcdef4a5bb62aef943c32c18a', 'request_id': 'req-c8c194c7-0c58-48db-b79c-9890094c7e99'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:35.309522 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] The token cache is being invalidate because project 1d786e2dcdef4a5bb62aef943c32c18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:20:35.309802 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:35.310433 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-c8c194c7-0c58-48db-b79c-9890094c7e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:35.311948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1288/2582] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:20:35 2022] DELETE /identity/v3/projects/1d786e2dcdef4a5bb62aef943c32c18a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:35.316767 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.317052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.317322 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.336344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:35.337533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:35.354224 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:35.401047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1295/2583] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:35.409255 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.409547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.409824 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.429720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:35.430915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:35.438732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1289/2584] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:35.442993 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.453154 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb167d2-5687-4efe-8e34-fe0854506834 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.457813 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.458407 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.458681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.458950 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.459631 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:20:35.459929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.461498 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.474227 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1296/2585] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:20:35.478130 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.489957 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b16f00-63c8-406b-b3aa-2c37c22dcec2 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.496202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.496922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.497353 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.497751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.498595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.500547 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.531653 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1290/2586] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:20:35 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:20:35.535836 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.546629 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca52b-8090-4080-8649-5e18d1e5d4cd admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.612049 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.612702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.612976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.613255 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.613943 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.616040 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.641710 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1297/2587] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:20:35 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:20:35.645577 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.659352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d76f-c01e-4bc8-b1a7-f6defca14c75 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.665319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.666080 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.666360 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.666631 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.667478 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.669226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.680362 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1291/2588] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:35.684387 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.694695 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d78df-2173-4f73-9fd2-62275ed89d93 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.699297 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e11938-81fb-447d-9cf5-32a32f64a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.699929 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e11938-81fb-447d-9cf5-32a32f64a373 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.700207 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e11938-81fb-447d-9cf5-32a32f64a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.700484 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e11938-81fb-447d-9cf5-32a32f64a373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.701685 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e11938-81fb-447d-9cf5-32a32f64a373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.703263 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e11938-81fb-447d-9cf5-32a32f64a373 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.713692 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1298/2589] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:35.718516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.728510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7cdb9-fb58-4435-be91-64621fcedef6 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.733158 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.733797 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.734069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.734331 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] PATH_INFO: `/v3/projects/f2cc06fd793047d88ae38ee9ba0dc160/users/7034ca2bc2314fec889ea9c3d1268178/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.748738 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2cc06fd793047d88ae38ee9ba0dc160, user_id=7034ca2bc2314fec889ea9c3d1268178, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.751534 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.764839 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1292/2590] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:20:35 2022] PUT /identity/v3/projects/f2cc06fd793047d88ae38ee9ba0dc160/users/7034ca2bc2314fec889ea9c3d1268178/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:35.776014 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.776303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.776578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.804917 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] MFA Rules not processed for user `7034ca2bc2314fec889ea9c3d1268178`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:35.806142 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:35.872383 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1299/2591] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:35.884251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:35.885921 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:35.908034 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:35.943636 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f9ff-f70b-434f-8dd3-a85209ea6298 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:35.947250 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea3e0a-3bcc-4387-8101-552fdb178d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463c2cf049454fa281e99b1a864cdf60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:35.947885 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea3e0a-3bcc-4387-8101-552fdb178d3b service cinder] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:35.948162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea3e0a-3bcc-4387-8101-552fdb178d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:35.948429 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea3e0a-3bcc-4387-8101-552fdb178d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:35.957283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea3e0a-3bcc-4387-8101-552fdb178d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:35.958716 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea3e0a-3bcc-4387-8101-552fdb178d3b service cinder] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:35.967692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1293/2592] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:44.569949 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:44.571408 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:44.590679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:44.638493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17db00-7b37-415e-80ca-aca82940a65b None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:44.643046 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:44.643808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:44.644129 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:44.644419 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] PATH_INFO: `/v3/users/06abb53024d04abbad308a8d91d09ff1` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:44.645533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] RBAC: Authorizing `identity:delete_user(user_id=06abb53024d04abbad308a8d91d09ff1)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:44.647524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:44.694881 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06abb53024d04abbad308a8d91d09ff1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:44.695560 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06abb53024d04abbad308a8d91d09ff1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:44.696155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06abb53024d04abbad308a8d91d09ff1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:44.703251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06abb53024d04abbad308a8d91d09ff1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:44.712038 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06abb53024d04abbad308a8d91d09ff1'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:44.730010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1300/2593] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:20:44 2022] DELETE /identity/v3/users/06abb53024d04abbad308a8d91d09ff1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:44.734982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:44.735778 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:44.735857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:44.755590 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:44.756834 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:44.828490 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1294/2594] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:44.838185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:44.839586 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:44.895796 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444741d-4d9c-451c-82f7-52576dc6afae admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:44.899269 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:44.899896 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:44.900184 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:44.900454 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:44.910153 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:44.911609 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:44.921833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1301/2595] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:45.268500 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:45.269913 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:45.282529 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd05d9c-b572-4b94-b8f5-f887a7464714 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:45.287704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:45.288340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:45.288616 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:45.288888 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] PATH_INFO: `/v3/projects/94ce1d1c6a1a49e494ff30ad588f661c` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:45.290010 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ce1d1c6a1a49e494ff30ad588f661c)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:45.292087 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:45.338643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ce1d1c6a1a49e494ff30ad588f661c', 'request_id': 'req-f0550379-167c-47f7-a96b-1c73fa218c5b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:45.345445 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ce1d1c6a1a49e494ff30ad588f661c', 'request_id': 'req-f0550379-167c-47f7-a96b-1c73fa218c5b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:45.352515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ce1d1c6a1a49e494ff30ad588f661c', 'request_id': 'req-f0550379-167c-47f7-a96b-1c73fa218c5b'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:45.363436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] The token cache is being invalidate because project 94ce1d1c6a1a49e494ff30ad588f661c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:20:45.364193 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:45.364827 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-f0550379-167c-47f7-a96b-1c73fa218c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:45.366288 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1295/2596] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:20:45 2022] DELETE /identity/v3/projects/94ce1d1c6a1a49e494ff30ad588f661c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:48.101543 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:48.102997 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:48.116615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55434dfa-ecf4-4bca-83ee-fe0281aee325 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:48.153972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:48.172886 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:48.173505 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.173780 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.174078 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] PATH_INFO: `/v3/users/7034ca2bc2314fec889ea9c3d1268178` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.175244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7034ca2bc2314fec889ea9c3d1268178)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:48.177170 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:48.221776 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7034ca2bc2314fec889ea9c3d1268178'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.222422 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7034ca2bc2314fec889ea9c3d1268178'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.223056 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7034ca2bc2314fec889ea9c3d1268178'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.229910 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7034ca2bc2314fec889ea9c3d1268178'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.237105 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7034ca2bc2314fec889ea9c3d1268178'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.254031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1302/2597] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:20:48 2022] DELETE /identity/v3/users/7034ca2bc2314fec889ea9c3d1268178 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:48.258520 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.258974 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.259064 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.278809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:48.280019 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:48.321181 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:48.340242 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1296/2598] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:48.350385 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:48.351795 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:48.408167 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e412d-632d-46fd-9e02-fe6059626e01 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:48.411797 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed7882-12b0-4294-bb0e-b103a0e877a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:48.412403 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed7882-12b0-4294-bb0e-b103a0e877a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.412679 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed7882-12b0-4294-bb0e-b103a0e877a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.412947 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed7882-12b0-4294-bb0e-b103a0e877a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.422024 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed7882-12b0-4294-bb0e-b103a0e877a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:48.423506 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed7882-12b0-4294-bb0e-b103a0e877a8 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:48.432719 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1303/2599] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:48.733809 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:48.735669 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:48.748900 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a72a88-b3b1-4684-97bc-ca937e776f83 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:48.754640 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:48.755692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.755692 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.755910 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] PATH_INFO: `/v3/projects/f2cc06fd793047d88ae38ee9ba0dc160` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.756933 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2cc06fd793047d88ae38ee9ba0dc160)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:48.758992 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:48.807908 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2cc06fd793047d88ae38ee9ba0dc160', 'request_id': 'req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.815544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2cc06fd793047d88ae38ee9ba0dc160', 'request_id': 'req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.823596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2cc06fd793047d88ae38ee9ba0dc160', 'request_id': 'req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.835785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] The token cache is being invalidate because project f2cc06fd793047d88ae38ee9ba0dc160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:20:48.836076 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.836728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-47ac9387-81a1-45d7-89f2-c08adcbb2a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:20:48.838223 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1297/2600] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:20:48 2022] DELETE /identity/v3/projects/f2cc06fd793047d88ae38ee9ba0dc160 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:48.843350 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.843656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.843930 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.863925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:48.865185 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:48.927821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1304/2601] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 07:20:48.936169 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.936453 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.936726 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.956964 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:48.958150 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:48.966141 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1298/2602] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:48.970358 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:48.975035 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:48.980887 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afff038-97aa-4923-9bce-ca9ee8a329a5 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:48.987183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:48.987842 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:48.988123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:48.988395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:48.989066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 07:20:48.989392 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:48.991016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.003273 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1305/2603] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Oct 11 07:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 07:20:49.007466 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:49.016657 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0407a-6e19-486c-95ef-2c62e5938993 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:49.021253 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:49.021841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.022117 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.022388 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.023024 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:49.025162 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.051489 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1299/2604] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:20:49 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:20:49.055683 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:49.065945 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b5635-b0e4-4bee-9883-94dfc363acc8 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:49.119664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:49.120508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.120508 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.120808 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.121386 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:49.123919 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.144678 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1306/2605] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:20:49 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:20:49.148730 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:49.158818 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af8858-808e-4389-8589-7ae2d3c68794 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:49.163328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:49.163948 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.164222 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.164516 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.165151 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:49.166752 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.177423 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1300/2606] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:49.181244 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:49.191741 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0418cb7-e9e3-4dc7-a88d-7a67d83165b6 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:49.196289 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcbdec-4aea-4e56-bc73-a5686bc8fee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:49.196877 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcbdec-4aea-4e56-bc73-a5686bc8fee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.197155 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcbdec-4aea-4e56-bc73-a5686bc8fee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.197426 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcbdec-4aea-4e56-bc73-a5686bc8fee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.198066 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcbdec-4aea-4e56-bc73-a5686bc8fee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:49.199681 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcbdec-4aea-4e56-bc73-a5686bc8fee4 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.210050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1307/2607] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 07:20:49.214116 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:49.223916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bf68e-9103-49f9-8e83-c47bc5f832ee admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:49.228432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:49.229065 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.229340 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.229610 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] PATH_INFO: `/v3/projects/709d4e3bf4de4bd6857171f2d5d77885/users/96a69fa70c78405d805f8b70860f57bf/roles/2985373b41e749088f96f5c6c9b092c8` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.244357 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=709d4e3bf4de4bd6857171f2d5d77885, user_id=96a69fa70c78405d805f8b70860f57bf, role_id=2985373b41e749088f96f5c6c9b092c8)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:49.247634 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.263836 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1301/2608] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:20:49 2022] PUT /identity/v3/projects/709d4e3bf4de4bd6857171f2d5d77885/users/96a69fa70c78405d805f8b70860f57bf/roles/2985373b41e749088f96f5c6c9b092c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:49.268846 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb290e8-bd04-46dc-94bb-8c4d5f0198eb None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.269132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb290e8-bd04-46dc-94bb-8c4d5f0198eb None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.269406 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb290e8-bd04-46dc-94bb-8c4d5f0198eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.288457 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-5fb290e8-bd04-46dc-94bb-8c4d5f0198eb None None] MFA Rules not processed for user `cdd55bd4e43346b29310a317dad7a19b`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:49.289657 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5fb290e8-bd04-46dc-94bb-8c4d5f0198eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:49.347062 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1308/2609] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:49.357302 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:49.359001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:49.415363 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2731f-1618-4459-83f6-637e19c9cee2 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:49.419649 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:49.420249 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:49.420555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:49.420826 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:49.429754 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:49.431178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:49.440382 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1302/2610] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:53.816652 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:53.817211 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:53.817428 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:53.860390 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] MFA Rules not processed for user `96a69fa70c78405d805f8b70860f57bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:53.861998 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:53.952127 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:53.959512 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1309/2611] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:53.962930 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:53.973108 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9cc3f-7e64-4699-a81f-2198746147e0 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:53.977728 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:53.978333 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:53.978607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:53.978871 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:53.979586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:53.981552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:53.987504 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:54.013001 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1303/2612] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Oct 11 07:20:53 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 07:20:54.016424 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:54.027606 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70def29c-2fc2-4d30-82e1-288cbaf350c9 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:54.100169 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:54.100968 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:54.101348 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:54.101723 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:54.102602 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:54.104662 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:54.136884 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1310/2613] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Oct 11 07:20:54 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 07:20:54.139463 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:54.151756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357021d-159e-4592-9ed6-4eb03412bc02 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:54.157869 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:54.159171 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:54.159595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:54.160256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:54.160787 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:54.163189 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:54.194370 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1304/2614] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:54.195572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:54.212664 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aecb9c-657e-4fd9-b81d-6229d7bd04c1 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:54.225529 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:54.226308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:54.226595 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:54.226875 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] PATH_INFO: `/v3/projects/65b91d4d173340399fa269c791573f4a/users/9bee52be8d9a445092eb9387744597ea/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:54.260572 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b91d4d173340399fa269c791573f4a, user_id=9bee52be8d9a445092eb9387744597ea, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:54.267552 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:54.282237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1311/2615] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:20:54 2022] PUT /identity/v3/projects/65b91d4d173340399fa269c791573f4a/users/9bee52be8d9a445092eb9387744597ea/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:54.286399 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:54.298070 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606c084-11b0-4f54-9f15-b531981ea4cd admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:54.384375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:54.384997 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:54.385341 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:54.385643 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:54.386324 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:54.387966 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:54.402789 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1305/2616] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Oct 11 07:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 07:20:54.403848 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:54.415030 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993d88d-d8ce-4d2c-aedf-c8d3b167a5cd admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:54.420921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea72c5-6069-4044-a8ef-7ac9128e18bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:54.422128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea72c5-6069-4044-a8ef-7ac9128e18bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:54.422128 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea72c5-6069-4044-a8ef-7ac9128e18bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:54.422518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea72c5-6069-4044-a8ef-7ac9128e18bf admin admin] PATH_INFO: `/v3/projects/65b91d4d173340399fa269c791573f4a/users/9bee52be8d9a445092eb9387744597ea/roles/287ae34730df480e9b5436bfab5d48bd` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:54.424696 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea72c5-6069-4044-a8ef-7ac9128e18bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b91d4d173340399fa269c791573f4a, user_id=9bee52be8d9a445092eb9387744597ea, role_id=287ae34730df480e9b5436bfab5d48bd)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:54.427352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea72c5-6069-4044-a8ef-7ac9128e18bf admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:54.439615 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1312/2617] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Oct 11 07:20:54 2022] PUT /identity/v3/projects/65b91d4d173340399fa269c791573f4a/users/9bee52be8d9a445092eb9387744597ea/roles/287ae34730df480e9b5436bfab5d48bd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:20:55.167359 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:55.169510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:55.249914 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89aad3-ae6a-4e53-beec-ca2dfa9890ed admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:55.304916 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:55.305596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:55.305875 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:55.306138 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:55.308501 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:55.320233 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:55.321940 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f86b50-9618-446b-89c6-e4d787b731e6 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:55.332971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1306/2618] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:55.764256 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312bf47-88f6-4a8e-a7a9-832091201805 None None] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:55.764781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312bf47-88f6-4a8e-a7a9-832091201805 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:55.764781 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312bf47-88f6-4a8e-a7a9-832091201805 None None] PATH_INFO: `/` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:55.766317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1313/2619] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 07:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 07:20:55.775107 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f910f-b8c9-4374-8f9e-9c86d747fb80 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:55.775608 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f910f-b8c9-4374-8f9e-9c86d747fb80 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:55.776018 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f910f-b8c9-4374-8f9e-9c86d747fb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:55.879454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-a41f910f-b8c9-4374-8f9e-9c86d747fb80 None None] MFA Rules not processed for user `0d9f52ea57a54084ac5aaf40f42dba58`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:55.881982 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a41f910f-b8c9-4374-8f9e-9c86d747fb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:55.897052 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1307/2620] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 07:20:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:55.930306 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312bf47-88f6-4a8e-a7a9-832091201805 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:55.956130 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312bf47-88f6-4a8e-a7a9-832091201805 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:55.961324 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c20634-e54d-4fb2-a958-a262f82443a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:55.961941 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c20634-e54d-4fb2-a958-a262f82443a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:55.962222 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c20634-e54d-4fb2-a958-a262f82443a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:55.962504 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c20634-e54d-4fb2-a958-a262f82443a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:55.970906 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c20634-e54d-4fb2-a958-a262f82443a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:55.972369 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c20634-e54d-4fb2-a958-a262f82443a6 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:55.981237 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1314/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:56.093650 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be053c-43f8-4f22-8a34-ed86a14b375e None None] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:56.094045 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be053c-43f8-4f22-8a34-ed86a14b375e None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:56.094459 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be053c-43f8-4f22-8a34-ed86a14b375e None None] PATH_INFO: `/` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:56.096482 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1308/2622] 10.222.0.39 () {54 vars in 921 bytes} [Tue Oct 11 07:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 07:20:56.102600 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5108fc-9603-4720-93ec-8d99a283b670 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:56.103050 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5108fc-9603-4720-93ec-8d99a283b670 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:56.103513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5108fc-9603-4720-93ec-8d99a283b670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:56.171841 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-6c5108fc-9603-4720-93ec-8d99a283b670 None None] MFA Rules not processed for user `fbf7156a9d9448a6987fe7999543cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:56.173084 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6c5108fc-9603-4720-93ec-8d99a283b670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:56.299926 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1315/2623] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Oct 11 07:20:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:20:56.309436 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be053c-43f8-4f22-8a34-ed86a14b375e None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:20:56.315551 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:20:56.328409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be053c-43f8-4f22-8a34-ed86a14b375e None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:20:56.337251 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:20:56.337868 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:56.338142 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:56.338407 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:56.364866 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:20:56.367307 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:20:56.383656 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1309/2624] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:20:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:20:59.901976 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c506b1-ad52-4f30-b352-ac39b627c69a None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:20:59.902555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c506b1-ad52-4f30-b352-ac39b627c69a None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:20:59.902555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c506b1-ad52-4f30-b352-ac39b627c69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:20:59.933052 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-c6c506b1-ad52-4f30-b352-ac39b627c69a None None] MFA Rules not processed for user `9bee52be8d9a445092eb9387744597ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:20:59.934925 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c6c506b1-ad52-4f30-b352-ac39b627c69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:20:59.939176 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:21:00.008561 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1316/2625] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:20:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:21:00.021232 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:00.022683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:00.089856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c13d2-dfcf-4be2-b7e8-9820e945384f service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:00.094717 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b826760f3d747b888ea373cf5021903', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:00.099736 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:00.100876 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:00.101562 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:00.181704 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:00.186856 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:00.201857 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1310/2626] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:21:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:21:00.356395 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974598ce-0d86-4b22-8274-bf007925b175 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:00.356711 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974598ce-0d86-4b22-8274-bf007925b175 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:00.356990 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974598ce-0d86-4b22-8274-bf007925b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:00.412009 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-974598ce-0d86-4b22-8274-bf007925b175 None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:21:00.413284 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-974598ce-0d86-4b22-8274-bf007925b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:00.484331 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1317/2627] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 07:21:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:21:00.532295 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:00.533785 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:00.627112 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818833c-e646-4fd4-9f02-c342a9d96f23 service glance] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:00.632487 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682756d1-a391-4c93-ab91-683374082575 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:00.633023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682756d1-a391-4c93-ab91-683374082575 service swift] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:00.633321 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682756d1-a391-4c93-ab91-683374082575 service swift] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:00.633596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682756d1-a391-4c93-ab91-683374082575 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:00.642449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682756d1-a391-4c93-ab91-683374082575 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:00.644256 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682756d1-a391-4c93-ab91-683374082575 service swift] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:00.653555 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1311/2628] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 07:21:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 11 07:21:09.704597 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] REQUEST_METHOD: `POST` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:09.705171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:09.705171 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:09.723622 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.auth.core [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] MFA Rules not processed for user `96a69fa70c78405d805f8b70860f57bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:21:09.724817 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:09.744493 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1318/2629] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:21:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:21:10.649561 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=94 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:21:11.810118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577326-2557-454c-ad7c-bf4cefb9ce6c None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:11.811050 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577326-2557-454c-ad7c-bf4cefb9ce6c None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:11.811328 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577326-2557-454c-ad7c-bf4cefb9ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:11.832985 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-6a577326-2557-454c-ad7c-bf4cefb9ce6c None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:21:11.834191 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-6a577326-2557-454c-ad7c-bf4cefb9ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:11.850633 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1312/2630] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Oct 11 07:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:21:11.866656 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:11.887515 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96cbee-8fb9-438a-a368-82f0ce94c891 None None] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:11.892268 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:11.892874 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:11.893151 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:11.893418 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:11.903894 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:11.905464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:11.914029 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1319/2631] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 07:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 07:21:11.986694 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:11.987016 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:11.987265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:12.025848 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] MFA Rules not processed for user `52173fbead6e4b499a99a6979c9a9898`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:21:12.028494 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:12.127508 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1313/2632] 10.222.0.39 () {58 vars in 958 bytes} [Tue Oct 11 07:21:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:21:20.630772 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:20.632366 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:20.664004 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:20.666283 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:20.670044 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=100 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:21:20.723108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61e903-82f4-43f1-8ccb-dabdfd053536 service swift] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:20.729789 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:20.730394 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:20.730673 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:20.730948 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:20.733272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:20.745535 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:20.746782 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14af0f-bb6b-43c4-8fb9-96aadee32ae5 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:20.747116 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:20.752592 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:20.753228 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:20.753502 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:20.753770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:20.757637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1320/2633] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:21:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:21:20.763384 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:20.765074 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:20.776336 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1314/2634] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:21:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 07:21:25.153031 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:25.154641 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:25.171132 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63f16-aaea-4f79-b008-ab58c41f3d31 service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:25.176785 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:25.177388 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:25.177665 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:25.177946 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:25.179725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:25.192725 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:25.194313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:25.205736 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1321/2635] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:21:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:21:29.678319 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:29.680406 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:29.697732 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df3150-6a73-4755-be7f-a79baf55e9b2 service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:29.703204 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:29.703835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:29.704115 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:29.704389 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:29.706106 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:21:29.716963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:29.718548 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:29.727842 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1315/2636] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:21:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:21:35.200589 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=27 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:21:39.622308 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:21:39.631962 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57d1c1-85d2-4de8-ad58-9b57c4be96ba service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:21:39.635771 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:21:39.636368 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:21:39.636663 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:21:39.637107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:21:39.646554 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:21:39.648015 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:21:39.657902 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1322/2637] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:21:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:21:39.723526 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=70 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:21:49.653178 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:05.875772 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:05.885308 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14893913-4ab8-4b3e-9f6a-b4cf91d64d26 service nova] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:05.889574 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:05.890285 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:05.890569 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:05.890841 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:05.892606 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:05.902510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:05.903996 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:05.913486 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1316/2638] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:22:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:22:15.909994 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=6 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:29.158431 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:29.173471 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb49c7-2586-4785-a446-b40cdcdbc1cc service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:29.180068 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbf7156a9d9448a6987fe7999543cedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:29.180676 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:29.181417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:29.181417 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:29.183027 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:29.195983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:29.197578 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:29.221321 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1323/2639] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:22:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:22:37.177229 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:37.206835 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ca22c-48ee-4282-808d-a50f8c93661f service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:37.211918 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:37.212515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:37.212788 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:37.213054 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:37.215030 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:37.241419 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:37.243995 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:37.257599 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1317/2640] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:22:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:22:39.205097 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:40.344229 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:40.366012 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acdbce-8021-4e7f-b5b7-c222f3461d03 service nova] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:40.370999 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:40.371701 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:40.371980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:40.372626 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:40.375513 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:40.392095 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:40.394338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:40.395207 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:40.397393 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:40.415621 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1324/2641] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:22:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:22:40.500902 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d5b2b-938a-476b-9a38-dd2cba1a0f6d service placement] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:40.505454 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:40.506079 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:40.506358 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:40.506630 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] PATH_INFO: `/v3/users/211be7ccf81649db928d8efd0a91426f` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:40.507823 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211be7ccf81649db928d8efd0a91426f)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:40.509753 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:40.554356 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211be7ccf81649db928d8efd0a91426f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.555007 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211be7ccf81649db928d8efd0a91426f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.562720 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211be7ccf81649db928d8efd0a91426f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.563595 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211be7ccf81649db928d8efd0a91426f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.571662 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211be7ccf81649db928d8efd0a91426f'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.588832 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1318/2642] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:22:40 2022] DELETE /identity/v3/users/211be7ccf81649db928d8efd0a91426f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:22:40.764317 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:40.774918 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aecb0b-efb2-4f1c-8a03-b408de6e166d service placement] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:40.833613 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:40.834599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:40.834599 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:40.834965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] PATH_INFO: `/v3/projects/0fa103dda8c34135af4c1a3913ccb564` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:40.836047 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa103dda8c34135af4c1a3913ccb564)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:40.838123 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:40.885833 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa103dda8c34135af4c1a3913ccb564', 'request_id': 'req-6781e052-a492-444a-8a38-6944fb4beccb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.894000 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa103dda8c34135af4c1a3913ccb564', 'request_id': 'req-6781e052-a492-444a-8a38-6944fb4beccb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.901416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa103dda8c34135af4c1a3913ccb564', 'request_id': 'req-6781e052-a492-444a-8a38-6944fb4beccb'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.911566 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] The token cache is being invalidate because project 0fa103dda8c34135af4c1a3913ccb564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:22:40.911850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.912503 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:40.913980 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1325/2643] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:22:40 2022] DELETE /identity/v3/projects/0fa103dda8c34135af4c1a3913ccb564 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:22:40.917931 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:40.920002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:40.932756 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed3c16-040c-4b2e-8bbd-83052ea66386 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:40.945439 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:40.991759 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:40.992385 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:40.992664 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:40.992936 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] PATH_INFO: `/v3/users/ab74860e0e5346c085bbb4f6e367d2a2` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:40.994110 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab74860e0e5346c085bbb4f6e367d2a2)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:40.996098 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:41.039862 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab74860e0e5346c085bbb4f6e367d2a2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.040456 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab74860e0e5346c085bbb4f6e367d2a2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.047810 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab74860e0e5346c085bbb4f6e367d2a2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.048449 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab74860e0e5346c085bbb4f6e367d2a2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.055279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-d7b77b38-9a71-4a97-bd25-46a77ca055eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab74860e0e5346c085bbb4f6e367d2a2'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.073032 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1319/2644] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:22:40 2022] DELETE /identity/v3/users/ab74860e0e5346c085bbb4f6e367d2a2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:22:41.243432 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:41.244984 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:41.260316 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e052-a492-444a-8a38-6944fb4beccb admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:41.317782 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:41.318338 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:41.318618 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:41.318892 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] PATH_INFO: `/v3/projects/051f15ffb9c44cafa699693c33e2b14a` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:41.320065 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051f15ffb9c44cafa699693c33e2b14a)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:41.322005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:41.329540 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:41.369144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051f15ffb9c44cafa699693c33e2b14a', 'request_id': 'req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.377161 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051f15ffb9c44cafa699693c33e2b14a', 'request_id': 'req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.384064 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051f15ffb9c44cafa699693c33e2b14a', 'request_id': 'req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.394251 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] The token cache is being invalidate because project 051f15ffb9c44cafa699693c33e2b14a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:22:41.394533 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.395144 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:22:41.396759 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1326/2645] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:22:41 2022] DELETE /identity/v3/projects/051f15ffb9c44cafa699693c33e2b14a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:22:51.069465 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:51.391010 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:22:58.353208 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] REQUEST_METHOD: `POST` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:58.353660 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:58.353758 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:58.373199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.auth.core [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] MFA Rules not processed for user `b0080cb54dc84d2da383b5d97bd64fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81485) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 07:22:58.374425 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:58.428715 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1320/2646] 10.222.0.39 () {58 vars in 967 bytes} [Tue Oct 11 07:22:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 07:22:58.468921 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:22:58.470352 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:22:58.539636 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcbe7f-bc20-42ab-ba0f-1087fa219d53 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:22:58.545107 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '498356fa9eaa41e09da95e0a151105dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:22:58.545703 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:22:58.545983 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:22:58.546257 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:22:58.554093 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:22:58.555697 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:22:58.563596 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1327/2647] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Oct 11 07:22:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 07:23:05.087963 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:05.089513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:05.160683 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d7d0f-0012-492a-b3f4-80e1a2faa7e0 None None] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:05.165279 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:05.165922 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:05.166202 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:05.166474 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] PATH_INFO: `/v3/users/96a69fa70c78405d805f8b70860f57bf` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:05.168226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a69fa70c78405d805f8b70860f57bf)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:05.170164 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:05.220513 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a69fa70c78405d805f8b70860f57bf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.221173 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a69fa70c78405d805f8b70860f57bf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.228178 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a69fa70c78405d805f8b70860f57bf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.228833 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a69fa70c78405d805f8b70860f57bf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.237197 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a69fa70c78405d805f8b70860f57bf'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.254746 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1321/2648] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:23:05 2022] DELETE /identity/v3/users/96a69fa70c78405d805f8b70860f57bf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:05.430005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:05.441187 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd40ab3-f1fe-442a-b25b-72fcad431abf service swift] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:05.498608 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:05.499235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:05.499545 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:05.499819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] PATH_INFO: `/v3/projects/709d4e3bf4de4bd6857171f2d5d77885` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:05.501016 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=709d4e3bf4de4bd6857171f2d5d77885)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:05.502987 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:05.548438 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709d4e3bf4de4bd6857171f2d5d77885', 'request_id': 'req-aa676fe4-9556-41f0-9a10-a9b787833a4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.556272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709d4e3bf4de4bd6857171f2d5d77885', 'request_id': 'req-aa676fe4-9556-41f0-9a10-a9b787833a4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.562580 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709d4e3bf4de4bd6857171f2d5d77885', 'request_id': 'req-aa676fe4-9556-41f0-9a10-a9b787833a4c'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.572720 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] The token cache is being invalidate because project 709d4e3bf4de4bd6857171f2d5d77885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:23:05.573021 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.573624 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.577531 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1328/2649] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:23:05 2022] DELETE /identity/v3/projects/709d4e3bf4de4bd6857171f2d5d77885 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:05.581544 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:05.582989 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:05.596461 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68a76-1f66-4455-9498-0680ca7cadb5 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:05.616897 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:05.654790 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:05.655409 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:05.655712 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:05.655980 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] PATH_INFO: `/v3/users/9bee52be8d9a445092eb9387744597ea` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:05.657089 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bee52be8d9a445092eb9387744597ea)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:05.659002 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:05.703462 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bee52be8d9a445092eb9387744597ea'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.704100 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bee52be8d9a445092eb9387744597ea'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.710934 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bee52be8d9a445092eb9387744597ea'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.711587 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bee52be8d9a445092eb9387744597ea'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.719355 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bee52be8d9a445092eb9387744597ea'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:05.737118 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1322/2650] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:23:05 2022] DELETE /identity/v3/users/9bee52be8d9a445092eb9387744597ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:05.910108 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:05.911754 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:05.924667 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa676fe4-9556-41f0-9a10-a9b787833a4c admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:05.941659 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:05.983702 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:05.984325 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:05.984601 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:05.984869 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] PATH_INFO: `/v3/projects/65b91d4d173340399fa269c791573f4a` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:05.985986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b91d4d173340399fa269c791573f4a)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:05.987972 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:06.032975 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b91d4d173340399fa269c791573f4a', 'request_id': 'req-c499962a-0ed9-4785-aa88-6d3da707a39b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:06.040003 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b91d4d173340399fa269c791573f4a', 'request_id': 'req-c499962a-0ed9-4785-aa88-6d3da707a39b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:06.046272 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b91d4d173340399fa269c791573f4a', 'request_id': 'req-c499962a-0ed9-4785-aa88-6d3da707a39b'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:06.056427 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] The token cache is being invalidate because project 65b91d4d173340399fa269c791573f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:23:06.056718 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:06.057873 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:06.059344 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1329/2651] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:23:05 2022] DELETE /identity/v3/projects/65b91d4d173340399fa269c791573f4a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:15.733661 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:16.053518 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:22.791199 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:22.792865 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:22.860348 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81438508-fc02-48e2-896e-85d46254bf02 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:22.908831 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:22.909434 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:22.909708 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:22.909975 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:22.911714 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:23.001596 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:23.002499 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:23.022855 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1323/2652] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:23:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:23:24.202727 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:24.204248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:24.216280 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499962a-0ed9-4785-aa88-6d3da707a39b admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:24.219819 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:24.220416 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:24.220693 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:24.220965 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:24.222637 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:24.236139 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:24.237555 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:24.248967 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1330/2653] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:23:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:23:33.020269 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=90 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:34.243527 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=6 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:36.023603 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:36.025028 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:36.036094 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ae96d-c6a9-4eaf-b574-f365edf18374 service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:36.039607 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9f52ea57a54084ac5aaf40f42dba58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:36.040194 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] REQUEST_METHOD: `GET` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:36.040484 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:36.040751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:36.043252 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:36.104069 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:36.106102 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:36.115535 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1324/2654] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:23:42.294011 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:42.295511 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:42.311323 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b936b33-1d87-4b57-a601-8054f911f093 service neutron] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:42.317865 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:42.318798 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:42.319146 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:42.319558 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] PATH_INFO: `/v3/users/52173fbead6e4b499a99a6979c9a9898` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:42.320821 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52173fbead6e4b499a99a6979c9a9898)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:42.322850 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:42.371005 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52173fbead6e4b499a99a6979c9a9898'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.371684 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52173fbead6e4b499a99a6979c9a9898'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.372291 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52173fbead6e4b499a99a6979c9a9898'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.379125 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52173fbead6e4b499a99a6979c9a9898'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.386970 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52173fbead6e4b499a99a6979c9a9898'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.408303 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1331/2655] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:23:42 2022] DELETE /identity/v3/users/52173fbead6e4b499a99a6979c9a9898 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:42.580620 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:42.591203 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597801a8-ed78-4c12-9a66-2995c969061a service neutron] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:42.654131 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:42.654751 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:42.655023 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:42.655290 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] PATH_INFO: `/v3/projects/60dd1d4ab8434282b5f57d6547f8d3e9` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:42.656414 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60dd1d4ab8434282b5f57d6547f8d3e9)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:42.658305 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:42.703552 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60dd1d4ab8434282b5f57d6547f8d3e9', 'request_id': 'req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.709770 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60dd1d4ab8434282b5f57d6547f8d3e9', 'request_id': 'req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.716586 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60dd1d4ab8434282b5f57d6547f8d3e9', 'request_id': 'req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.726602 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] The token cache is being invalidate because project 60dd1d4ab8434282b5f57d6547f8d3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:23:42.726889 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.727510 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.729176 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1325/2656] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:23:42 2022] DELETE /identity/v3/projects/60dd1d4ab8434282b5f57d6547f8d3e9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 07:23:42.733313 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:42.734722 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:42.748410 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda024db-bd2b-4ee2-afbe-333423ac1a95 admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:42.802292 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:42.802940 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:42.803235 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:42.803514 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] PATH_INFO: `/v3/users/1261192c1b724bc8a0d020c5df609800` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:42.804635 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1261192c1b724bc8a0d020c5df609800)` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:42.806550 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:42.852815 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1261192c1b724bc8a0d020c5df609800'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.853464 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1261192c1b724bc8a0d020c5df609800'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.854072 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1261192c1b724bc8a0d020c5df609800'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.861183 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1261192c1b724bc8a0d020c5df609800'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.868248 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.notifications [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1261192c1b724bc8a0d020c5df609800'} {{(pid=81484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:42.876467 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:42.887826 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1332/2657] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Oct 11 07:23:42 2022] DELETE /identity/v3/users/1261192c1b724bc8a0d020c5df609800 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:43.055971 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Authenticating user token {{(pid=81485) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:23:43.057472 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.fernet_utils [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81485) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:23:43.071049 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed871b1-7967-4f2e-97a1-e58fdf1ce766 admin admin] Validating token access rules against request {{(pid=81485) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:23:43.093432 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:43.132192 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdd55bd4e43346b29310a317dad7a19b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b69df7fd16f47b18a624f52e5487b78', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81485) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:23:43.132812 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:23:43.133085 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:23:43.133352 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] PATH_INFO: `/v3/projects/ee49cf21898f4f5894d61abbb9996acc` {{(pid=81485) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:23:43.134442 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee49cf21898f4f5894d61abbb9996acc)` {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:23:43.136915 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] RBAC: Authorization granted {{(pid=81485) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:23:43.182416 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee49cf21898f4f5894d61abbb9996acc', 'request_id': 'req-84113002-8656-4c2a-87ad-1a6b094e6a62'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:43.188444 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee49cf21898f4f5894d61abbb9996acc', 'request_id': 'req-84113002-8656-4c2a-87ad-1a6b094e6a62'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:43.195375 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee49cf21898f4f5894d61abbb9996acc', 'request_id': 'req-84113002-8656-4c2a-87ad-1a6b094e6a62'} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:43.206226 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] The token cache is being invalidate because project ee49cf21898f4f5894d61abbb9996acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81485) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 07:23:43.206515 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:43.207139 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG keystone.notifications [None req-84113002-8656-4c2a-87ad-1a6b094e6a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81485) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 07:23:43.208644 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: [pid: 81485|app: 0|req: 1326/2658] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Oct 11 07:23:43 2022] DELETE /identity/v3/projects/ee49cf21898f4f5894d61abbb9996acc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 07:23:52.888409 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:23:53.203265 dv-cinder-854954-3-iscsi devstack@keystone.service[81485]: DEBUG dbcounter [-] [81485] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81485) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 07:26:22.994080 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Authenticating user token {{(pid=81484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 07:26:22.995487 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:26:23.074986 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab749-b496-4f8c-a3e6-150d86768cce admin admin] Validating token access rules against request {{(pid=81484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 07:26:23.132604 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d6eb8eb03314f6aa7fe0ba171babf57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89e92b29982040fe9ee65a3a0bae8af8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 07:26:23.133210 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] REQUEST_METHOD: `GET` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 07:26:23.133490 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] SCRIPT_NAME: `/identity` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 07:26:23.133762 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 07:26:23.135524 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.fernet_utils [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 07:26:23.146922 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 07:26:23.148532 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dbe73-aafe-48c0-949b-7f9e1465a5fe service placement] RBAC: Authorization granted {{(pid=81484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 07:26:23.158907 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: [pid: 81484|app: 0|req: 1333/2659] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Oct 11 07:26:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 07:26:33.154553 dv-cinder-854954-3-iscsi devstack@keystone.service[81484]: DEBUG dbcounter [-] [81484] Writing DB stats keystone:SELECT=63 {{(pid=81484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}